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SUMMARY 



The mathematical theory of differential games is used to study the 
structure of optimal allocation strategies for some time-sequential 
combat games with combat described by Ranches ter- type equations of warfare. 
Although most of this work concerns the application of existing theory for 
the determination of optimal campaign strategies in various Lanchester- 
type differential games of tactical interest, theoretical developments are 
also given consideration. 

Previous research by the author had uncovered an important gap in the 
existing theory of differential games: no adequate theory of state variable 

inequality constraints existed for differential games. Such a theory is 
essential for Ranches ter-type differential games because, for example, of 
the requirement that force levels be non-negative. Consequently, in this 
report first-order necessary conditions of optimality are developed for 
differential games with state variable inequality constraints. These results 
are then applied to the study of optimal compaign strategies in several 
differential games of tactical interest. 

Optimal air-war campaign strategies are studied in a generalization 
of a well-known tactical air-war model. Previous work had never given 
adequate consideration to necessary conditions of optimality with respect 
to the non-negativity of aircraft force levels. The work at hand extends 
previous work by considering a value for survivors at the end of the air 
war and variable (effectiveness) coefficients. Several specific problems 
are solved. The effects of temporal variations in the return from ground 
support on optimal campaign strategies are studied. 
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The differential game model discussed above has been criticized 
because it does not evaluate air-war tactics within the context of ground- 
war objectives. Consequently, this model produces suboptimization. Hence, 
a model is considered which is considerably broader in scope, does consider 
land-war objectives, but (unfortunately) is considerably less susceptible 
to closed-form analytic solution. For this new model partial results 
concerning the determination of optimal campaign strategies are obtained 
and contrasted with those for the model previously discussed above. 

The determination of optimal fire distribution strategies for 
supporting weapon systems is a major problem of military operations 
research. The author has studied some aspects of this problem in previous 
research. Motivated by a paper which recently appeared in the open liter- 
ature, optimal fire-support strategies are studied in an attack scenario. 
The dependence of optimal fire-support strategies on combatant objectives 
is examined for a problem previously considered in the open literature: 
necessary and sufficient conditions on the functional form of the criterion 
functional (terminal payoff only) for the optimal fire-support strategies 
to be independent of force levels (at least for a certain range of values) 
are developed. This shows that certain results previously reported in the 
literature are, in general, not true. A variation of this problem is 
then considered. In contrast to the previous work reported in the litera- 
ture, the attrition structure of the problem at hand leads to the optimal 
fire-support strategy of the attacker requiring him to sometimes split his 
artillery fire between enemy infantry and artillery (counterbattery fire). 
Numerical examples are given. 



11 



Finally, some problems for possible future study are discussed. 

The formulation of these problems is based on the author’s past research 
experiences. As is usually the case, work on the problems considered 
here and elsewhere has uncovered other aspects of interest that appear 
to merit further examination. 
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1. Introduction. 



This report is the third in a series of reports which document 
the author *s research on the structure of optimal (time-sequential) 
allocation policies for tactical situations described by Lanchester- 
type equations of warfare. By considering several combat scenarios, 
further insights have been gained into such important questions as: 

(1) How should fire be distributed over targets? 

(2) How should targets be selected? 

(3) Do target priorities change over time? 

(4) Do force levels affect the optimal allocation strategies? 

(5) How does the number of target types affect the optimal 
allocation strategies? 

(6) Do conflict termination circumstances affect the optimal 
allocation strategies? 

(7) How are the optimal fire distribution/ target selection 
strategies affected by the nature of the attrition processes? 

(8) What is the effect of logistics constraints on such policies? 

(9) How does the uncertainty and confusion of combat affect 
optimal allocation strategies? 

Specific objectives of this research have been the identification 
of decision parameters and the further development of general principles 
for target selection, regulation of firing rate, and allocation of 
military resources in dynamic combat situations. A further discussion 
of research objectives and their relationship to defense planning 
problems can be found in the author’s past reports* [11], [12]. 

;Ar 

A comprehensive review of pertinent literature prior to 1973 in the 
fields of (a) Lanchester- type models of warfare, (b) differential games, 
and (c) optimal control of Lanchester-type attrition processes is also 
to be found in [12]. 



It should be pointed out that although the quantitative analysis of tactics 
may be considered to be a relatively new field of military operations 
research, there is wide-spread interest in such work (see [6], [8], [9], 

[10], [19]). 

Our approach has been to combine Lanches ter-type models of warfare 
with generalized control theory (both deterministic and stochastic optimal 
control, dynamic programming, and differential games) [12], [15] (see also 
[16]). This has been done by considering a sequence of concrete problems. 
Additionally, the report at hand contains some basic research on the 
theory of differential games. Past research [12], [15] had identified as 
an important gap in the existing theory of differential games the lack 
of adequate theory (i.e. necessary conditions of optimality) for problems 
with state variable inequality constraints (SVIC*s). 

The organization of this report is to discuss results in general 
terms in the main body and to leave details for the appendices. This 
has been partially necessitated by the wide scope of problems examined 
(from theoretical to applied). 

2 . Guided T our of the Ap pend i ces . 

In this section we summarize the work which is contained in the 
appendices and explain why this work was done. The results reported 
here may be considered to be extensions of our earlier work [11], [12], 

[15]. Moreover, the work at hand lays the foundations for more extensive 
work on the quantitative analysis of tactics and applications of generalized 
control theory to problems of military operations research. 
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In Appendix A we develop a theory of state variable inequality 
constraints for Lanches ter-type differential games. First order 
necessary conditions of optimality are developed for the class of problems 
in which individual state variables are bounded by functions of time. 
Although this certainly is not the most general type of state variable 
inequality constraint (SVIC) , it does include all Lanchester- type 
differential games that occur in military operations research. First 
order necessary conditions of optimality are also stated (but not proved) 
for more general problems. In particular, results are given without proof 
for problems with a fairly general type of SVIC. The proofs of these 
results will be given in the future. 

Recently, we pointed out [12], [15] that no adequate multiplier 
conditions existed for (Lanchester-type) differential games with state 
variable inequality constraints and gave this top research priority. 

It is indeed remarkable that essentially all previous applications of 
the theory of differential games to Lanchester-type problems (see, for 
example, [1] , [2] , [4] , [5] , [7] ) are inadequate in one respect or another 
with respect to SVIC’s. A further discussion and substantiation of 
this claim is to be found in Appendix A. This work establishes the 
mathematical foundations for future applications of the theory of dif- 
ferential games because force levels must be, for example, non-negative 
(see [17] for a discussion within the context of Lanchester-type optimal 
control problems) . 

In Appendices B and C we apply the theoretical results of Appendix 
A to the study of optimal air-war strategies. This has been over the 
years a problem of lasting interest for defense planners. In Appendix B 
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we consider a generalization of the tactical air-war game of A. Mengel 
[41 (see also [3]) in which aircraft effectiveness may vary over time and 
a residual value for aircraft at the end of the campaign is considered. 
Optimal strategies are characterized for this general problem. A 
complete solution (i.e. explicit determination of optimal closed-loop 
strategies) is given for cases of constant coefficients. These results 
are much more general than previous ones and provide much insight into 
tradeoffs between various planning parameters. Preliminary results are 
also obtained for problems in which aircraft effectiveness in the air 
war is constant but the value of ground support changes over time. The 
latter is an attempt to reflect the influence of combat air support on 
the ground war and hence avoid suboptimization. Two cases of time- 
dependent '^returns’* from ground support are considered: 

(a) linearly-decreasing-with- time returns from ground support, 

(b) exponentially-decreasing-with-time returns from ground support. 

One criticism that has been made of the model considered in 

Appendix B is that it does not evaluate air-war strategies within the 
context of ground-war objectives. In other words, this model (unfor- 
tunately) produces suboptimization. Thus, in Appendix C we present 
preliminary results for a model which considers the development of 
optimal air-war strategies within the context of land-war objectives. 

These preliminary results show that the outcome of the ground war may 

be a significant factor in the determination of optimal air-war strategies. 

The research reported in Appendices B and C was undertaken because 
an important question for defense planners is what are appropriate 
missions over the course of a campaign for tactical air power. The 
answer to this question has far-reaching implications for Navy air forces 
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(both carrier-based and land-based) and, of course, the Air Force. 

Recently, the USAF Studies and Analysis Group has been using quantitative 
methodology [19] in trying to answer such questions. By considering the 
results given in Appendices B and C one can begin to see the effects of 
the nature of the combat optimization problem on the structure of optimal 
air-war strategies . 

The determination of optimal fire distribution strategies for 
supporting weapon systems is a major problem of military operations 
research. This problem is particularly of interest to the military 
tactician so that he may have a clearer understanding of the circum- 
stances, for example, under which enemy infantry should be engaged by a 
supporting weapon system (such as artillery) and those under which "counter- 
battery” fire is to be preferred. Furthermore, the study of such tactical 
time-sequential allocation problems is relevant to the Navy mission of 
fire support (both by ship gunfire and by carrier-based air). 

In Appendix D we further consider the determination of optimal fire 
distribution strategies for supporting weapon systems. In this case it 
is appropriate to call the supporting weapon system artillery. We consider 
extensions of some recent work by Kawara [5] concerning optimal strategies 
for supporting weapon systems in an attack scenario which is a variation 
of a model considered by Weiss [20]. We first examine for what class of 
criterion functionals (terminal payoff only) the optimal fire-support 
strategies are independent of the force levels and then examine the depen- 
dence of the optimal fire-support strategies upon the form of the combat 
attrition model by considering slightly different combat dynamics than 
those considered by Kawara [5]: we assume that the attacker ^s artillery 
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produces "linear-law” attrition (see [13], [16]) against both the defender’s 
artillery as well as infantry. The development of a complete solution 
to the latter problem has involved solution phenomena not previously 
encountered in a Lanchester- type differential game: the dual variables 

are discontinuous across a manifold of discontinuity of both players* 
strategies . 

The research reported in Appendix D was undertaken to obtain a 
better understanding of the dependence of the structure of optimal 
fire-support strategies upon model form. Previous work by Weiss [20] 

(see also [4]) and Kawara [5] had given the impression that an optimal 
fire-support strategy consists in always concentrating all fire on one 
enemy target type. From our previous work [13] on a one-sided 
Lanchester- type, time-sequential fire distribution problem in which each 
of two enemy target types undergoes a **linear-law'* attrition process, we 
knew that an optimal fire distribution policy could consist in splitting 
one*s fire between available target types. This was our motivation for 
the examination of other attrition structures in Kawara* s problem. 
Additionally, Kawara *s recent work [5] stimulated our research on the 
influence of combatant objectives on optimal fire-support strategies 
because of Kawara *s remarkable result that (at least within a certain 
range of force levels) optimal fire-support strategies do not depend on 
force levels. We were able to show that Kawara had used the only type of 
objective function that yeilds this result. Thus, in general, the result 
is not true. 

In Appendix E we briefly describe several important problems for 
possible future research. These problems were formulated giving 
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consideration to our past research experiences (both that reported here 
and also elsewhere). These problems may be referred to as 

(a) optimal fire support for several ground units, 

(b) a Lanchester-type optimal control problem with logistics 
considerations , 

(c) an examination of the influence of the form of the criterion 
functional on optimal strategies in time-sequential combat 
optimization problems. 

As explained in Appendix E, work on these campaign analysis problems 
would appreciably enhance our understanding of methodology for the 
optimization of combat dynamics. 

^ • Summary of Research Findings . 

Here we summarize our research results. We have (at least partially) 
accomplished the following tasks which were suggested for future research 
in our previous report [12]: (a), (b2), (e), (g)*, (il), (j2), and (k) . 

Results are organized under the following headings: 

(1) solution methodology for time-sequential combat games, 

(2) insights gained into the optimization of combat dynamics, 

(3) implications for defense planning. 

Items (2) and (3) differ in that the latter is a management-oriented 
digest of the practical implications of our research, whereas the former 
is oriented towards a technical audience. Further amplification of 
results and conclusions is to be found in the appendices. 

a . Solution Methodology for Time-Sequential Combat Games . 

Our research has produced the following results on solution 
methodology for time-sequential combat games. Specifically, we have 
accomplished the following: 



‘^See [18]. 
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(1) developed theory of state variable inequality constraints 
for Lanchester- type differential games (This theory is 
essential for all problems with, for example, negativity 
restrictions on force levels.)* 

(2) demonstrated use of theory of state variable inequality 
constraints for solving time-sequential combat games by 
developing solutions to several specific combat optimiza- 
tion problems, 

(3) developed co mplete solution to a fire support differential 
game. This involved the following technical difficulties: 

(a) singular surfaces, 

(b) discontinuity in the dual variables, 

(4) developed methodology for determining the functional form 
of the criterion functional (terminal payoff here) which 
yields optimal strategies that do not depend on force 
levels (at least for a certain range of force levels), 

(5) concluded that computational methods must give consideration 
to structural properties of optimal strategies in idealized 
versions like those considered in this report. 

b . Insights Gained into the Optimization of Combat Dynamics . 

Based on our study of the optimization of combat dynamics using 
the mathematical theory of differential games, we have reached the following 
conclusions : 

(1) the structure of optimal strategies depends on the following 
factors : 

(a) criterion functional, 

(b) combat attrition model, 

(c) battle termination model; 

the dependence is complex, and future research should concen- 
trate upon the examination of numerous structures of 
tactical interest, 

(2) optimal fire-support strategies depend on the nature of the 
attrition process produced by the supporting weapon system; 
when the supporting weapon system produces casualties at a 
rate proportional to the product of the numbers of firers 
and targets of a particular type (see Appendix D for 
mathematical formulation), the optimal fire-support strategy 
of the supporting weapon system has the following character- 
istics: 
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(a) it depends on the optimal strategy for the enemy’s 
supporting weapon system, 

(b) the optimal distribution of supporting fires may be 

(i) to concentrate on enemy infantry, 

(ii) to split fire to avoid overkilling, 

(iii) to concentrate on enemy artillery, 

(c) counterbattery fire is the optimal strategy during 
the early stages of an attack and destruction of 
enemy infantry is the optimal strategy during the 
final moments (unless the ratio of defender infantry 
to defender artillery is ’’extremely” large, in which 
case the infantry is always engaged), 

(d) a split of supporting fires between enemy infantry 
and artillery as an optimal strategy can only occur 
when enemy infantry have some effectiveness against 
friendly infantry, 

(3) optimal air-war strategies are different for the case 
in which ground war objectives are considered than for 
that in which they are not, 

(4) optimal air-war strategies (see Appendix B for scenario) 
depend on the following factors: 

(a) residual value of surviving aircraft, 

(b) force levels (especially when a side is going to 

lose the fight for air supremacy: faced with a 

future loss of airpower, one may abandon the fight 
for air supremacy and get what ground support one 
can from his planes before he loses them all), 

(5) cases with variations over time in weapon system 
effectiveness are best studied as extensions of constant 
coefficient cases . 

c . Implications for Defense Planning . 

In our research reported here we have studied idealizations 
of allocation structures that commonly occur in defense planning studies. 
After studying these idealizations in order to gain insight into the 
structure of optimal strategies in the complex real-world problem, we 
have reached the following conclusions as to considerations that should 
be brought to the attention of defense planners. These results should 
be kept in mind by practitioners who perform more detailed computer 
simulation studies . 
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(1) The combat optimization problem should be thought of as 
consisting of three parts: 

(i) combatant objectives, 

(ii) conflict termination conditions, 

(iii) combat dynamics. 

Optimal combat strategies depend on all three of the 
above. More basic scientific research should be done on 
all three, especially the first two. 

(2) Force levels do affect (either directly or indirectly) 
optimal combat strategies. 

(3) It may be quite dangerous to generalize optimal combat 
strategies developed for specific problems. At present, 
more research is needed on specific problems in order to 
develop an understanding of the qualifications that may 
be necessary to make about specific study results. 

(4) Optimal air-war strategies must be based on ground-war 
objectives. Suboptimization results when this is not 
done. This suboptimization is a serious problem, since 

it may lead to winning the air campaign but losing the war. 

4. Su ggested Future Research Task s. 

After performing the research documented in this report, we feel that 
the current state-of-the-art for applying the theory of differential games 
to time-sequential combat games is such that more significant results 
may be readily obtained in the future. Moreover, this previous research 
provides valuable perspective for identifying what appear to be the most 
important research tasks to be considered next. In our opinion the most 
important task is to examine (and then understand) the influence of objec- 
tives on optimal strategies. Another important task is to study the 
structure of optimal fire-support strategies. 

Based on our past research experience we feel that there is much to be 
accomplished in the future. Specifically, we suggest the following as 
future research tasks: 

(a) Examination of the effects of campaign objectives (as reflected 
by the criterion functional) on the structure of optimal campaign 
strategies. In all our past research we have with one exception 
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always considered a linear utility for survivors in the criterion 
functional. It is of interest to examine how the valuation of 
survivors affects the structure of optimal strategies. Other 
criterion functionals that might be considered are 

(1) nonlinear valuation of survivors (either the difference 
or ratio of functions which are, for example, concave, 
quasi-concave, etc , ) , 

(2) the value of losses rather than survivors. 

It would seem appropriate to begin such an investigation by con- 
sidering the simplest problem possible, 

(b) Further study of optimal fire-support strategies. The actual 
military operations must be analyzed and decisions identified. 

For various cases, the appropriate scenario and, consequently, 
Lanchester-type model would be developed (see Appendix E) , and 
the optimization problem studied. It is felt that at this time 
it is most important to study various specific problems in order 

to gain insight into the structure of optimal fire-support strategies. 

(c) Examination of the effects of logistics constraints on optimal 
campaign strategies. Models would be developed to relate logistics 
capability to combat effectiveness and then appropriate combat 
optimization problems formulated. Such research would provide 
insight into the worth of the Navy logistics (pipeline) role in 
combat service support missions (see Appendix E) , 

(d) Further study the determination of optimal air-war strategies 
within the context of land-war objectives. Based on preliminary 
results given in this report, it appears as though the outcome of 
the ground war is a significant factor* in the determination of 
optimal air-war strategies and that optimal strategies developed 
for a model not considering land-war objectives need not be optimal 
for a model which does consider them. This proposed research 
would extend the results given in Appendix C of the report at hand. 

A more comprehensive (although somewhat dated now) discussion of 
suggested future research tasks can also be found on pages 65-67 of our 
previous report [12], Work still remains to be done on the following 
tasks suggested there: (b), (c), (d) , (f), (h), and (j). 



*This is not considered, for example, in TAG CONTENDER [19], 
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APPENDIX A: Necessary Conditions of Optimality for Differential Games 

with State Variable Inequality Constraints. 

1 , Introduction . 

In our most recent report on applications of differential games to 
problems of military conflict [22] we suggested as a future research task 
that a theory of state variable inequality constraints (SVIC’s) be developed 
for Lanchester-type differential games (see [23]). This suggestion was 
based on our discovery of an important gap in the existing theory of 
differential games: no adequate treatment of multiplier conditions for 

differential games with SVIC's then existed in the literature. As we have 
pointed out previously [23], [25], state variable inequality constraints 
are present in all Lanchester-type dynamic tactical allocation problems 
because force levels (which are represented by state variables ) are required 
to be non-negative (or some equivalent condition). Thus, the development 
of a theory of SVIC’s is essential for an adequate characterization of an 
optimal strategy in such a problem (see examples given in Appendix B) . It 
is indeed remarkable that this important gap in theoretical results has 
not even been noted by other workers in operations research. 

The origins of the theory of differential games are given in [10]. 
Since the publication of this paper in 1965 and that of the highly signifi- 
cant book by Isaacs [11] , numerous papers have been published on the subject 
of differential games. Further references can be found in [22]. Necessary 
conditions of optimality have been developed for differential games by 
Isaacs [11] and Berkovitz [3], [4]. In two highly technical and significant 
papers [3], [4], Berkovitz developed necessary conditions for problems 
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subject to various types of inequality constraints except those only 
involving functions of the state variables (i.e. except SVIC^s). Varia- 
tional methods were used in [3] by reducing the problem to two associated 
variational problems. In [4] the properties of the value function were 
utilized to develop these conditions by use of Isaacs* *'tenet of transi- 
tion.” Schmitendorf and Citron [16] extended Berkovitz*s variational 
approach to second order conditions and developed a conjugate point condi- 
tion for a class of differential games. Later, Schmitendorf [15] used the 
same approach to develop (both first and also second order) necessary 
conditions of optimality for (zero-sum deterministic) differential games 
in which one or both players are restricted to use open-loop control. The 
development of necessary conditions has also been disucssed by Chattopadhyay 
[7]. 

In this appendix we will develop necessary conditions of optimality 
for differential games with SVIC*s by an extension of the approach of 
Berkovitz [3]. The development of necessary conditions for such problems 
has apparently not been accomplished before although sufficient conditions 
for optimality have been derived by a number o f workers [13] , [18] . 

Quite independently of the above developments A. Friedman began a 
careful study of the existence of value for a differential game and existence 
of a saddle point in pure strategies. This work led to the book [9] and 
has continued. Differential games with SVIC * s have been considered by 
Friedman, but he has focused primarily on the question of existence of 
value for them (see Chapter 6 of [9]). We have used Friedman*s results 
in our past work [19], [20], [21]. However, Friedman [9] does not develop 
the two-sided analogues of well-known control theory multiplier conditions 
for constrained subarcs which lie on the boundary of the state space [12] . 
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In fact, the treatment of the example on pp. 239-240 of [9] is inadequate, 
since the approach fails to yield optimal strategies for minor modifications 
in the problem's formulation (see Appendix B) . Moreover, this problem is 
very important for military operations research (see Appendix B) . 

The research reported in this appendix is of a more basic nature 
than that reported elsewhere in this report. These results are also of 
much wider application than to the problems of defense planning upon which 
the author has concentrated. Moreover, the derivation of these results 
allows one to identify the class of problems to which such multiplier 
conditions apply. It should be noted that not all system dynamics give 
rise to such conditions. 

2 . A Lanchester-Type Differential Game . 

In [23] we coined the term Lanchester-type differential game as 
referring to a differential game in which the system dynamics are described 
by Lanchester-type equations of warfare. To be precise, we are considering 
two-person zero-sum deterministic differential games in which each player 
uses a closed-loop (or feedback) strategy. A formulation which is rele- 
vant to our research on tactical allocation problems is as follows: 

n m 

maximize minimize { w.y.(t^) - ^ v.x.(t^)}, 

, i=l ^ ^ ^ i=l ^ ^ ^ 

ij iJ 

Stopping rule: t^ - T = 0, 

dx . n 

subject to: — = r. - 7 il/..a..y. for i = l,...,m, 

(combat dynamics) ^ ^ j=l ^ 

dy , m 

= s. - y d)..b^.x. for i = l,...,n, (1) 

dt 1 ij j > > > 



with 



x^,y^ ^ 0 for all i,j , 



m 



y }h . . ^ 1 for j = 1 , 
i=i 



y (f) . . 1 for j = 1 , 



i=l 



ij 



(State Variable Inequality Constraints), 

,n, (Strategic 

Variable 
Inequality 
Constraints) 



where 



(f) . . ,i|^. . ^ 0, 

ij ij 



and 



x^(t), y^(t) are force levels, 





S . 

1 


are 


replacement rates , 






w. 

1 


are 


the utilities assigned survivors. 


a . , 
ij 


is 


the 


rate at which one Y. unit 

J 


can destroy X^, 


b. . 
ij 


is 


the 


rate at which one X. unit 

J 


can destroy Y^, 


4'. . 

ij 


is 


the 


fraction of X. who fire at 
J 




. 

ij 


is 


the 


fraction of Y, who fire at 
J 


X. . 

1 



The above problem (1) contains all the essential features (and can be 
made identical to by minor changes in formulation) of all the Lanchester- 
type differential games studied previously in the operations research 
literature (see [23]). 

It should be noted that we use capital letters to denote a strategy 
(see [3] or [9] for precise definitions), whereas the corresponding lower 
case letter is used to denote the realization of the strategy (Friedman 
uses the term outcome (see pp. 22-23 of [9]). A strategy is a mapping 



from the state space to the space of admissible decisions. Heuristically , 



it is a contingincy plan that tells us what to do depending upon where we 
find ourselves in the state space (time may sometimes be an additional 
state variable). In other words, we have = 4^(t,x,y) where ^ denotes 

-k 

a q vector, x denotes an m vector, and ^ denotes an n vector. 
Then strategic variable is the realization (or outcome) of a strategy as 
follows : 

tp. . (t) = f (t,x,y) . (2) 

ij ij ^ ^ 

As mentioned above, we will refer to (1) as a Lanches ter-type 
differential game. This suggests that we consider the following problem 
(denoted as ’’Problem I”) for the development of necessary conditions of 
optimality for a class of differential games with SVIC’s, 



Problem I, 



maximize minimize G(T ,x(T) ,y (T) ) + 
U V 






L(t,x,y ,u,v)dt , 



row 



subject to; 



X = f (t,x,y,v) , 









u(t), v(t) are unrestricted (scalar) strategic variables, 



x^(t) ^ a(t) for all t 6 [0,T] (scalar state 

variable inequality 

y^(t) ^ g(t) for all t 6 [0,T] constraints), 

with scalar terminal condition F(T,x(T) ,y(T)) = 0, 



k 

We use standard notation for vectors, etc. as defined, for example, on 
pp . 4-6 of [8] . 
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where we assume that all functions are smooth enough to insure the 
existence of all partial derivatives required in the following analysis. 

In Problem I above, x is an m-vector of state variables, and y is an 
n-vector of state variables. 

3 . Mathematical Preliminaries . 

In this section we further establish our notation and state our 
assumptions. We consider closed-loop (or feedback) strategies , denoted 
as U(t,x,y) and V(t,x,y), for the players. A strategy then is the 
specification of what value should be given to a player’s strategic variable 
at each point in time depending upon the time and current values of the 
state variables. The realization (or outcome) of a strategy will be 
denoted by the corresponding lower case letter. Thus 



We call u(t) and v(t) strategic variables . 

The criterion functional is denoted by J, and thus we have for 
Problem I 



Player I chooses U(t,x,y) and tries to maximize J, while Player II 
chooses V(t,x,^) and tries to minimize J. We say that (U*,V*) is a 
saddle point of J if 



u(t) = U(t,x,y) and v(t) = V(t,x,y). 



(3) 



T 



J = G(T,x(T) ,y(T)) + 



L(t,x(t) ,y(t) ,u(t) ,v(t))dt. 



(4) 



0 



J(u,V*(t,x,y)) J(U*(t,x,y) ,V*(t,x,y)) ^ J (U* (t ,x ,y) ,v) . 




(SPC) 



Then (U*,V*) is the solution to Problem I. We denote the value of the 



differential game as W. Then 

W = J(U*,V*). (5) 

We assume that Problem I has value and a saddle point in pure strategies. 
It will be convenient to denote the vector of state variables as 

T T T 

Z = (x ,y ) = (x , . . . ,x ,y , . . . ,y ) . (6) 

^ ^ ^ ± mi n 

We will then sometimes write terms in a more convenient form as 

L(t,z,u,v) = L(t,x,y,u,v) . (7) 

Although not essential for the development of necessary conditions of 
optimality, to insure the existence of value for the differential game we 
assume 

L(t,z,u,v) = L (t,x,y,u) + L (t,x,y,v). (8) 

It is assumed that both players have perfect information as to the 
state variables. In other words, both players know precisely the current 
state of the system. 

We assume that the first time derivative of y (t) explicitly 

n 

contains the strategic variable u and that (y ) = g (t,x,y,u) 4 - 0 

n n ^ 

u u 

along an optimal trajectory. We further assume that the first time 

derivative of explicitly contains the strategic variable v and 

that (x ) = f (t,x,y,v) 4 0 along an optimal trajectory. In other 

m m ^ 

V V 

* 

Several other technical conditions must be satisfied to guarantee the 
existence of value and of a saddle point in pure strategies to Problem I 
(see Chapter 6 of [9]). 



words, we assume that ^ ot(t) and ^ 3(t) are the appropriate 

* 

types of first order SVIC*s (see [12]). 



4 . Statement of Necessary Conditions of Optimality for Problem I . 
First we define 



H(t,z,p,q,n ,u,v) = L(t,z,u,v) + p f(t,x,y,v) 



9. - Hi (t){x^-a (t) } - n-,(t) {y^-6(t) }, (9) 



m 



n 



where 



and 



and 






= 0 


for 


X > a(t) 
m 


^ 0 


for 


X = a(t) 
m 



ri2(t) 



= 0 


for 


y < 6(t) 
n 


^ 0 


for 


y„ - 6(t) 



<I>(T,z(T),a) = G(T,x(T),y(T)) + aF(T,x(T) ,y (T) ) 



( 10 ) 



Let x*(t) for 0 ^ y ^ T denote the optimal path which results from 
(U*(t,x,y), V*(t,x,y)). We assume that the solution to Problem I is normal 
(see [1]) and that the optimal path is not tangent to the terminal manifold 

or to any manifold of discontinuity of U* or V* or to the boundary of 

the state space at entry to a constrained subarc which lies on the boundary. 
Then, in order that the strategy pair (U*,V5*f) be a saddle point of the 
criterion functional it is necessary that there exist unique functions 

In optimal control theory one says [6] that a problem has a p — order 

SVIC when the pill time derivative of the state-variable constraint is 

the first to explicitly contain the control variable. 
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p(t), q(t), n, (t) , and rio(t) and constants 0, v. , and such 

that the following conditions hold 







|H = 0 

dU 


9L 

or — 

dU 


4. T 

+ q 


9g 

9u 


0. 




(11) 






|H = 0 

dV 


9L 

or — 
9v 


4. T 

+ E 


9f 

9v 


0. 




(12) 




•T 

“P 


X 9x 


T 

+ P ^ 

^ 9x 


4. T 

+ q 


9g 

9x 


^ .m T 
n, (A ) , 
1 


(13) 




•T 

-q 


- H - ^ 

I ■ 'X 


T 


4 . T 

+ a 


' 


/ . n. T 
H2(A ) , 


(14) 


T 

P (T) 


9$ 


- V. (a”)^ 


where 




= 0 


for 


x^(T) > o(T), 
m 


(15) 


9x(T) 


^ 0 


for 


x^(T) = a(T),* 


T 

q (T) 


9'!' 


, * n>. T 
- v.(A ) 
z ^ 


where 


/ 


= 0 


for 


y^(T) < e(T), 


(16) 


8X.(T) 




2: 0 


for 


y (T) = e(T), 
n 






H(T) + If 


+ v^a(T) 


+ V. 


,8(T) 


= 0, 




(17) 



where 




0^ 




and 6 . . 
ij 



1 for i = j , 
0 otherwise. 



'k 

These results hold only when the boundary of the state space (at least the 
part for which x = a(t)) is non-absorbing (see [24]). For an absorbing 
state boundary we^ave 



/ = 0 


for X 




m 


V, / ^ 0 


for X 


1 \ 


m 


{ unrestricted 



m 



but X (t) > a(t) 
in 

= a (T) and x ( t) 
in 

t^ ^ t ^ T with 



for t < T, 

= a(t) for 
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Analogous to the Weierstrass condition, we also have the max-min principle 

H(u,v*) ^ H(u*,v), (18) 

which holds for all admissible u and v. H, J^(t), 

and q(t) are continuous (vector-valued) functions except at manifolds 
of discontinuity of both U* and V*, where for all differentials dt, 
dx, and dy along the manifold 

)dt - (p"*"-p )^dx - (q'*'-q )^dy = 0. (19) 

In other words, at both entrances to and exits from constrained 
subarcs on the boundary of the state space , we have 

£(t") = £(t^), (20) 

aVT) = £(A) , (21) 

and 

H(t')=H(A), (22) 

except possibly at manifolds of discontinuity of both U* and V^, where 
all the dual variables are continuous except those corresponding to 
or y^ which describes the boundary. 

The above first order necessary conditions of optimality for Problem 
I_ are analogous to the optimal control theory results of Jacobson, Lele, 
and Speyer [12]. The above results may also be written in a form analogous 
to that of optimal control theory results of Berkovitz [2] and Gamkrelidze 
(see Chapter VI in [14]). This is done by introducing new multipliers 
A(t) such that 

p(t) = A (t) - p (t)A"', (23) 

q(t) = A (t) - p (t)A'^. 

^ z ^ 



( 24 ) 
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It is well known (see [24]) that we may then make the following identifications 



n^(t) = -M^(t) for i = 1,2. 



(25) 



5 . Development of Necessary Conditions of Optimality . 



It is convenient to re-write Problem I as (DG) . 

T 

L(t,x,y,u,v)dt} , 



maximize minimize {G(T ,x(T) ,y(T) ) + 
U V ~ ~ 



subject to: 



X = f (t,x,y,v) , 
y = g(t,x,y,u) , 

r>^ 



(DG) 



u(t) , v(t) are unrestricted (scalar) strategic variables, 

X (t) ^ a(t) for all t € [0,T] (scalar state variable 

m 

inequality constraints) , 

y^(t) ^ S(t) for all t € [0,T] 

with scalar terminal condition F (T ,x(T) ,y (T) ) = 0. 

Considering the saddle point condition (SPC) for the criterion functional, 
necessary conditions of optimality for (DG) will be developed according 
to the approach of Berkovitz [3] in which maximum and minimum problems 
associated with the game are considered. Necessary conditions of optimality 
are developed for each of these two problems. It is then shown that these 
two sets of conditions may be written as a single set of necessary condi- 
tions for (DG) . 

We extend Berkovitz ’s variational approach to differential games 
with SVIC’s by treating boundary segments by two different methods in the 
associated optimal control problems. For the simple case of bounded state 
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variables at hand, we may think of both methods of treating an SVIC as 
techniques in which the dimensionality of the state space is reduced for 
a constrained subarc which lies on the boundary for a finite interval of 
time . 



5.1. Necessary Conditions for Control Problem A . 

Thus, we are led to maximum and minimum problems associated with 
(SPC) . The associated maximum problem we denote as Control Problem A , 
(CPA). We are to determine u’^fCt) = U’*f(t,x,y) such that the following 



holds : 



maximize {G (T ,x(T) ,y (T) ) + 
u(t) 



L(t,z,u,V*(t,x,y))dt}, 



subject to: x = f (t ,x,y , V* (t ,x ,y) ) , 

0^ 0^ 

y = g(t,x,y,u) , 



(CPA) 



u(t) is unrestricted scalar control variable, 

X (t) ^ a(t) and y (t) ^ g(t) for all t € [0,T], 
m n 

with scalar terminal condition F (T ,x (T) ,y (T) ) = 0. 

For (CPA) we assume that the first time derivative of y (t) 

n 

explicitly contains the control variable u and that (y ) = g (t,x,y,u) 

n n ^ 

u u 

4 0 along an optimal trajectory. Let us note that in (CPA) we have 

(x ) =0. Thus, variations in the control u have no direct influence 

m 



u 

on a trajectory for which x (t) = a(t). Let us recall that we have 

m 

assumed that in (DG) the first time derivative of explicitly con- 

tains the strategic variable v and that (x ) (t,x,y,v) 4 0 along an 

m ^ ^ 



V 
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optimal trajectory. Thus, in (CPA) we will have (since in (CPA) x (t) ^ 

m 

• • 

a(t) is a "rule of the game") x = f (t,x,y,V*(t ,x,y)) ^ a(t) when 

m m ^ ^ ^ 

X (t) = a(t) where f (t"^) denotes a right hand limit, 
m m 

There are two cases to be considered in developing necessary condi- 
tions of optimality for (CPA) ; 



(1) ^ a(t) almost everywhere in time, 

X X XX 



(2) x^(t) = a(t) 

m 



for t ^ t ^ t 

entry exit 



. t m ^ m 
with t < t , . 

entry exit 



CASE (1): X (t) > a(t) almost everywhere in time. 

We will apply the results of Speyer [12] (see also [17] and [24]) 
to (CPA). To this end, we define 



H^(t,z,p^,q^,u) = L(t,z,u,V*(t,z)) + (p^)^f (t , z ,V*(t , z) ) 



+ (q^)^g(t,z,u) - n,(t){y -6(t)}, (26) 

^ ^ ^ / n 



and 



where 

! = 0 for y < 6(t) , 

S: 0 for y = 6(t) , 
n 

$^(T,z(T),a^) = G(T,x(T) ,y(T)) + o^(T ,x(T) ,y (T) ) . 



(27) 



According to the results of Speyer [12] , first order necessary conditions 
of optimality for (CPA ) are as follows: 



9u 



= 0 



or 



9L , . A,T 

^ (‘I ) 

OU ^ 




(28) 



(pA)T . . It _ (pA)T _ A T !!: ,9L ^ A T ft „ 

9x ^ 9x ^ 9x ^9v X 



( 30 ) 



A- 



,-A,T 3L , A,T , A^T (3L , , A.T , .,n,T 

<a ) " - 5^ - (£ ) ^ - <3. > ^ + <£ ) 3?1 “£ ■" "2‘3 ) ■ 



(p\l))"- 



3x(T) 



A,.m.T , A / 

- V, (A ) where < 

1 ^ 1 ( 



= 0 for X (T) > a(T) , 



m 



0 for X (T) = a(T) , 
m 



(30) 



(31) 



(q (T))" = 



3<1> 



3y(T) 



A,.n.T , / 

- V (A ) where v. 



= 0 for y (T) < e(T) , 



^ 0 for y^(T) = e(T), 



(32) 



A ^ A • A • 

h'^(T) + H- + a(T) + 6(T) = 0, 



(33) 



where H^(T) is an abbreviation for H^(t=T,z(T) ,p^(T) ,q^(T) ,u*(T)) . We 
will make copious use of such notational conveniences. We also have the 
maximum principle 



H^(u,V*(t,z)) H^(u*,V*(t,z)) . 



(34) 



A A A 

H (t) , and ^ (t) are continuous except at manifolds of discontinuity 
of V*, where for all differentials dt, dx, dy along the manifold 

(H^‘^-H^")dt - (p^%^")^dx - (q^'^V')^dy = 0. (35) 



When y (t) < B(t) almost everywhere in time, the above becomes 



n 



(except at manifolds of discontinuity) 



3g 



BL ^ Av T ' — ^ 

<a> i;;- 



(36) 



As noted for equation (15) , these results hold only when the boundary of 
the state space (at least the part on which y^ = 3(t)) is non~absorb ing 
(see [24]). The reader is referred to the previous footnote for the state- 
ment of terminal multiplier properties for an absorbing state boundary. 
Henceforth, in order to contain the length of this appendix we will leave 
it to the reader to supply such necessary qualifications for conditions 
on terminal multipliers. 
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•A,T 



•A,T 



BL _ 


/ A,T 


9f 


/ A.T 


9g 

r*>^ 


Bx 

/>w> 


■ (p ) 


Bx 


• (q ) 


Bx 


9L 


/ A^T 


9f 


/ A,T 


!lr 




■ (£) 




• (q ) 


9y 

<v# 



■3L 



•8L 



A,T 



A, T 



3f 

df 

T^v*. 

9v^ y 



(37) 



(38) 



Using (36) , we may write the adjoint equations as 



,-A.T 3L , A,T , A,T ;9L , , A.T 



dx 



8x *■ Bu 



Bu** X 



Bf 



- f|^ " 3?! 



(39) 



•A.T 9L , A.T , A.T ;9L , . A.T 

(a, ) = - i 7 - (p > i 7 - (a, ) ^ 

rw /V/ ^ 



- 11^ + (P*)’'^ V*. 
"•Bv ^ Bv* ^ 



Bf 



(40) 



Y Y Y Y 

When y (t) = 6(t) for t ^ ^ t ^ t with t ^ < t , 

•^n entry exit entry exit 

we may reduce the dimension of the state space and also the adjoint variable 
space (see [17]). Let us adopt the following notation 



P = 



(P-l 5^9 > • • • 5 

1 z m 



i. - Vl>' 



-T 

g 



(^l’®2’ ‘ ' ‘ ’®n-l^ ’ 



-T 



I = (yi.y2-*-.Vi^’ 



-T / T -T. , 

z = (x ,^) = (x^, 



5 ^ 5 y -I ) • • • 5 y -1)5 

m 1 n-1 



V = V(t,z) = V(t,x,y,y = 6(t)). 
^ ^ ^ n 



In other words, in the reduced state space the state variables are 

^1’ ‘ ‘ ' *^m’^l’ * * ’ ’^n 1’ corresponding adjoint variables are denoted 

A A 

as P and Q . When V* is continuous at entry to the boundary of the 



state space with y = B(t), the corner conditions at such junctures to 

n Y 

this reduced state space are (for example, at entry with t = t ) as 

^ e entry 

follows for a first order SVIC [24] 
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(42) 

and 

H^(0=H^(t^). (43) 

e e 

Assuming that the optimal path is not tangent to the boundary of the state 

space at entry to the constrained subarc (i.e. g (t ) - 3(t ) ^ 0) , it 

n e e 

follows that 



L(t^) L(tj + (Q^(t^))^{i(t^) -g(t^)} 
A. -V 6“ e ^e e 

q (t ) = 



(44) 



where we have adopted the notation 



and 



t^A, a. 

p (t ) = p (t ) , 
0 0 



^A> +v -A. +v 

Q (t ) = q (t^) 



(45) 



When V* is discontinuous at entry to the boundary of the state space with 
^n ” follows from adapting the argument of Berkovitz [1] on 

discontinuous ’’right-hand sides” to the usual development of the corner 
conditions (see [24]) 



q^(t ) = q^(t^) + pA^, (47) 

and 

H^(r) = H^(t^) + pg(t^). (48) 



By (46) through (48) and tho non-tangency assumption, it follows that 
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, L(t^) - L(tj + (P^(t^))^{f(t^) - f(t^)} + Q^(tt))^{i(tt) - g(t^)} 

q (t ) = ::^ : (49) 

gn(te) - 6 (V 

where again we have used (45) . It should be noted that (49) reduces to 
(44) when V* is continuous at entry. We may also re-write (46) and (47) 

(or (41) and (42)) as 



and 



A, -V ^A. 
p (t ) = p (t^) , 
^ e ^ e 



“A.. -V -A. +v 

q (t ) = Q (t ). 
^ e ^ e 



(50) 



The adjoint equations in the reduced state space may be obtained from 
(29) and (30) by use of (28) to eliminate q^(t). From (28), we have 

9g 



A _ 1 i 3L ^ ,-A,T 

"^n 3g I 3u ^ 3u/’ 



3u 

since we have assumed ^ 0 along an optimal trajectory for (CPA). 

-A A 

Since q (t) is denoted as Q (t) in the reduced state space, this 
becomes 



A _ 1 /3L 

^n 3g^ ( 3u ^ 

3u 






(51) 



Using (51) and denoting the adjoint variables in the reduced state space, 
(29) may be written as 



•A,T 



(p")‘ . .It. (p;-). ^ 



,A,T 



3f 



.A,T 



^g 



9g. 



3x I A T 

ilr - ( 2 ,*)" 3 ?) 



3u 



. (li+ V*. 

^3v ^ 3v^ X 



( 52 ) 



A-18 



Similarly for (30) 






A.T 3L ,^A,T T r 3L , AT % 

) - + 1— + (Q ) J^\ 



(Q")- = _ — _ (P ) (Q 



dy 



dg ^ ^du 
°n 



du 



- ^ “■- • 

'•9v ^ dv^ y 



(53) 



It should be noted that, to be precise, in the reduced state space we have 



l^_3V 



-) _ 

k t,y 

rs^ 

X . for i=l , . 
^ i k 



= (— ) 

'•3x ^ 



k t,y 



X . 
1 



for i=l, 

± ^ k 



,m 



etc. However, to avoid excessive notation we will not be precise on this 
point and hope that the identification of the independent variables when 



partial derivatives are to be taken is clear from the context. When 

Y Y Y Y 



y (t) = g(t) for t ^ ^ t ^ t with t < t , we have 

n entry exit entry 



'exit^ 



Y Y 

y = g (t,x,y,y = 3(t) ,U’^(t,z)) = 3(t) for t ^ ^ t ^ t 

•'n n ^ ^ entry exit 



except at manifolds of discontinuity of U*(t,z) (i.e. except at the end 
points of the interval) . Differentiating the above identity with respect 
to X, we obtain 

r<^ 



I + g U* = 0, 
m ®n X 
X u 



which yields 



U 

X 







9u 



(54) 
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since 



3u 



^ 0 along an optimal trajectory. Similarly we obtain 



* ^ 

— TV 

U- = - ^ 

^ n 



du 



(55) 



Using (54) and (55) , we may write (52) and (53) as 






Hi 

9x 



A T 

^ 



w'')" ? - (f + ^ U* 



*-dv ^ 3v^ X 



(56) 






9L 

9y 



A T 

e ) ^ 

9y 



A T 

(Q ) — 

~ 9y 



r 9L 

^9v 



+ (Q^)^ A U- 
‘’9u 9u^ ^ 



3L 

-h 

A T -* 

(P^)^ ^ V- . 
^ 3v^ y 



(57) 



X X XX 

CASE (2); X (t) = a(t) for t ^ t :i^ t with t ^ < t . 

m entry exit entry exit 

We consider the optimal control problem (CPA) in a reduced state 

space in which the state variables are x., , . . . ,x ^ . ,y (also 

1 m-1 -^1 -^n 

-T -T T 

denoted as z = (x ,y )) and the corresponding adjoint variables are 
A A 

denoted as P , Q , Then the system dynamics are given by 



X = f (t,x,x =a(t) ,y,V (t,z)), 
in 



y = g(t,x,x =a(t) ,y,u) , 

^ ^ in 

—•k — — — 

with V (t,z) = V*(t,x,x =a(t) ,y) being determined by f (t,x,x =a(t) , 

^ ^ m ^ m ^ m 

^ _ 

y,V (t,z)) = a(t). We also will denote U(t,x,x =a(t) ,y) as U = U(t,z). 
^ ^ m 

In the reduced state space we define a Hamiltonian as follows 
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-* 



H^(t,z,P^,Q^,u) = L(t,x,x =a(t) ,y,u,V (t,zj)) 

TTl 



+ (P^)^f (t,x,x =a(t) ,y,V*(t,z)) 

#>x/ Itl 



,A,T 



+ (Q ) g^(t,x,x^=a(t) ,^,u) - n2(t){yj^-B(t) } 



(58) 



where 



( = 0 for y^ < 8(t) , 

;,(t) 

V ^ 0 for y^ = 3(t) • 



According to the results of Speyer [12], first order necessary conditions 
of optimality for (CPA) in CASE (2) are as follows: 

9g 



n ^ ,-A,T „ 



(59) 



(pA)T ^ ^ _ (pA^T ^ ^ A^T + (pA^T ^ ^ 

OX ox ox 



(qA)T (pA)T ^ ^ . (|i p (pA)T ^ -* ^ 

<s/ dy ^ dy ^ dy *‘dv ^ ov*^ y 2 

^ 



Using (59), we may write (60) and (61) for y < 3(t) as 

n 



(p^)'^ = - — - (p^)’^ 
8x ^ 



3f 



dx 






dx 



X 



- ||^ + (l'')'" ^ "z . 



Sv** X 



(62) 



(Q^" = 



iL 

3y 



(P^)^ 



3f 

3y 



- (Q*)* l|^+ (Q*)* ^ 5 

dy '‘du ^ du* y 



_ J|L ^ pa i ^ _ 

^8v ^ 3v^ y 



^A,T -* 



(61) 



( 63 ) 
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-A A A 

H , P (t) , and Q (t) are continuous except at manifolds of 

discontinuity of V* and possibly at junctures to the boundary of the 

state space with x = a(t). Thus, we also must consider necessary condi- 

m 

tions for a juncture between an unconstrained subarc and a constrained 
subarc on the state boundary. The results of Speyer and Bryson [ 17 ] tell 
us that 

p^(t^) = p(tj, 

and 



_A. +. 

Q (t^) = q (t ) , 
^0 ^0 



( 64 ) 



X 

whGre t = t and t dGnotes a laft hand limit. It also may he 

0 Gntry g 



shown that 



- p^(t )a(t^). 

0 0 m 0 e 



( 65 ) 



This may be written as 

L(t ,x,x =a(t ) ,y,u*(t^) ,V (t^,z)) + (P^Ct^))"^? (t ,x,x =a(t ) ,y,V (t\^)) 
e#^m 0^ 0 0^ -^0 ^e^m e ^ e ^ 



+ (Q^(t^))^g(t ,x,x =a(t ) ,y,u*(t^)) = L(t ,x,x =a(t ) ,y ,u*(t ) ,V (t ,z^)) 
^e^'^G^m e^ e e^m g^ g g^^ 



+ (p^(t ))^f (t ,x,x =a(t ) ,y,V*(t ,z)) + (q^(t ) )^g ( t ,x ,x =a (t ) ,^,u* (t ) ) 
^G^G^m e 0^ ^G^G^m e ^ e 



- p\t )ci(t ) , 
m 0 0 



( 66 ) 



or (using ( 64 )) 



(t_.z)) - “(t-)} 
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L(t ,x,x =a(t ) ,y,u*(t^) ,V (t^,z)) - L (t ,x ,x =a (t ) ,y ,u* (t ) ,V* (t , z) ) 



+ (P^(t^))^{f (t ,x,x =a(t ) jy.V^t^.z)) - f (t ,x ,x =a (t ) ,y ,V*(t , z) ) } 

+ (Q^(t'*'))^{g(t ,x,x =a(t ) ,y,u*(t^)) - g (t ,x,x =a (t ) ,y ,u* (t ) ) } . (67) 

^G >^G^m G^ G ^G'^m G^ G 



When the statG trajectory does not enter the boundary of the state space 

— — — — • 

tangentially, we have that f (t ,x,x =a(t ) ,y,V (t ,z)) - a(t ) 0, 

me-^m e^ e 

A - 

so that (67) may be solved (uniquely) for p (t ) . Then 

m e 



, L(t^)-L(tj+(P^(t^))^{f(t^)-f(t )}+(Q^(t^))^{g(t^)-g(tj) 

A, —V e e^e ^e^e ~e ^e^e 

p (t ) = 

me ^ / ”n • / N 

f (t ) - tx(t ) 
me e 



When f (t ) - a(t ) ^ 0 and u*(t ) = u*(t ) (i,e, no manifold of 
m e e e e 

discontinuity of at entry to reduced state space) , (68) becomes 



, L(t'^) - L(t ) + (P^(t‘^))^{f(t^) - f(t )} 

p (t ) = ;; < 

^ ^ f (O - a(t ) ■ 

me e 



(69) 



_ — :S: — _ • 

When f (t ,x,x = a(t ) ,y,V (t ,z)) - a(t ) = 0, then the juncture 
me^^m e^ e 

A - 

conditions (64) and (65) (i,e, (67)) do not determine p (t ) . 

m e 

To summarize, we have for (CPA) 



for ^ g(t), B(t) almost everywhere in time ; 

,.A,T 3L , A.T , A-T ^ /9L ^ / A^T * 



j3L , , A.T , * 



( 68 ) 



(70) 
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•A,T 

(q ) = 



8L / i ~ / ~ _L / A. i ^ 



A,T 



3f 



A,T 



9g 



9g 



- (|^ ^ <E*)" 



» 



(71) 



for y (t) = S(t) for t 

* n 



«ntry^ 



^ t ^ t 



-exit- 



with t 



-entry- 



< t 



-exit 



-m- 



;A,T 



8x 



•A T 

(Q ) = 



iL 

3y 



,A,T 



,A,T 



3f 


At 

<a")" ^ 


3x 




(3L 




■ (3v 


3£ 


A T 

<a ) ^ 


8y 


3y 




r3L 

Uv 



9g 



^ ()v ) X 



-■k 

U 

X 



IL 

• 8u 



A T k 

^ du^ y 



^ yt 

- ^ 3v^ y 



(72) 



(73) 



X 



m 



m 



m 



m 



for X (t) = a(t) for t ^ t ^ t with t ' < t . 

m entry exit entry exit 

and y^(^) ^ 3(t) almost everywhere in time ; 




r>^ 



( 75 ) 
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5.2. Necessary Conditions for Control Problem B . 

We also consider a minimum problem (denoted as Control Problem B , 
(CPB)) associated with (SPC) . We are to determine v*(t) = V*(t,x,y) such 
that the following holds: 



minimize {G(T ,x(T) ,y (T) ) 
v(t) 



+ 



T 

f 

L(t,z,U*(t,x,y) ,v)dt}, 

0 



subject to: 




t,x,y,v) , 



(CPB) 



y = g(t,x,y,U*(t,x,y)) , 

Aw/ 



v(t) is unrestricted scalar control variable, 



X (t) ^ a(t) and y (t) g(t) for all t 6 [0,T], 
m n 



with scalar terminal condition F (t ,x , (T) ,^(T) ) = 0. 



For (CPB) we assume that the first time derivative of x (t) 

m 

explicitly contains the control variable v and that (x ) = f (t,x,y,v) 

m V m ^ 

V 

^ 0 along an optimal trajectory. Let us note that in (CPB) we have 



(y ) =0. Thus, variations in the control v have no direct influence 

n V 

on a trajectory for which y (t) = 6(t) . Let us recall that we have 

n 

assumed that in (DC) the first time derivative of y (t) explicitly con- 

n 

tains the strategic variable u and that (y ) = g (t,x,y,u) ^ 0 along 

n u n ^ 

u 

an optimal trajectory. Thus, in (CPB) we will have (since in (CPB) 

* * 

^ 3(t) is a "rule of the game") y = g (t,x,y,U*(t ,x,y)) ^ B(t) when 

n n ^ ^ 

yn(t) = B(t) where g^(t"*") denotes a right hand limit. 

There are two cases to be considered in developing necessary condi- 



tions of optimality for (CPB) : 
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(1) y (t) < B(t) almost everywhere in time, 
n Y ' Y 

n ^ . n 



(2) y„(t) = B(t) for t 



n 



entry 



^ t t 



exit 



with t 



< t 

entry exit 



CASE (1): 3(t) almost everywhere in time . 

As before, we will apply the results of Speyer to (CPB) . To this 
end, we define 



H^(t,z,p^,q^,v) = L(t,z,U*(t,z) ,v) + (p^)^f (t,z,v) 






rw rw 



where 



and 



+ (q^)^g(t,z,U*(t,z)) - n-,(t){x -a(t)}, (76) 

- - - 1 m 



/>-> 






= 0 for X > a (t) , 
m 



^ 0 for X = a (t) , 
m 



<K®(T,z(T) ,a®) = G(T,x(T),y(T)) + a^F(T ,x(T) ,y(T) ) . (77) 

According to the results of Speyer [12], first order necessary conditions 
of optimality for (CPB ) are as follows: 




( 80 ) 
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B T 9$ 
(p (T))' = 



B 



9x(T) 



(q®(T))^ = 



9y(T) 



B 

V, (A ) where v, 
i ^ 1 



B, rivT B 

v^(A ) where 
z ^ z 



. B^ 

9T + V 



^2^ 



for 


X (T) > a(T), 
m 


for 


x^(T) = a(T) , 


for 


y^(T) < a(T), 


for 


y^(T) = 6(T), 


= 0. 





(81) 



(82) 



(83) 



We also have the minimum principle 



H®(U*(t,z) ,v) S: H®(U*(t,z) ,v*) . 



(84) 



B B B 

H , p (t) , and q (t) are continuous except at manifolds of discontinuity 



of U*, where for all differentials dt, dx, d^ along the manifo 



Id 



/_,B+ „B— V j . B+ B— vT . B+ B— V T . ^ /oc\ 

(H -H " ( 3 , “3, ) ^ 



When X (t) > a(t) almost everywhere in time, the above becomes 
m 



(except at manifolds of discontinuity) 



9L , , B.T _ 

(P ) 9^= 0’ 



( 86 ) 



,-B.T 
(P ) = 


_ 9L _ 


R T 

<^P ) T7 ■ 

^ dX 


, B.T 


r3L . 


9x 


) 17 ■ 

^ dX 


XT' + 

'• 9u 


/•B^T 


9L 


, B,T 


B.T 


f9L . 


(q ) = 


' " 






du 



9uJ x’ 



(87) 



( 88 ) 



Using (86) , we may write the adjoint equations as 



,-B^T 8L , B.T , B.T *8, ,3L . , E.T 

<E ) - - s - (Jt ) ST - 1 + (a ) ^ « 



- {fr + a < . 



(89) 



A-2 7 



,'B,T 3L , B.I ~ , B.I ,3L , B.T , ,* 

<a > ■ - ^ - ‘p > i7 - (a ) ^ + <a ) “y 



_ ||t + (pB)I ^ / . 

X/ Bv* y 



X 



• 9v 

X 



(90) 



X 



X 



When X (t) = a(t) for t :S t ^ r with t < t , we 

m entry exit entry exit 

may reduce the dimension of the state space and also the adjoint variable 
space (see [17]). Let us adopt the following notation 



P = (P P ...,P ), 

^ 1 Z m-1 



Q = (Q Q ...Q ), 
^ 1 Z n 



f^ = (f ,f ...,f ), 

^ 1 Z m-1 



-T 

X = (x ,x , . . . ,x ) , 
1 Z m-1 



U = U(t,z) = U(t,x,x =a(t) ,y) 

rw ^ m ^ 



In other words, in the reduced state space the state variables are 

corresponding adjoint variables are de- 

B B 

noted as P , Q . When U* is continuous at entry to the boundary of 



the state space with x = a(t) , the corner conditions at 

m 

to this reduced state space are (for example, at entry with 
as follows for a first order SVIC [24] 



such junctures 

X 



t 

e 



m 



'entry 



) 



B ^ *“v B , 4“v 

P )> 


(91) 


B/ “*>. B/ +x 

q (t- ) = q (t ) , 
^ 0 ^0 


(92) 


H^tJ) = H^t/). 


(93) 



Assuming that the optimal path is not tangent to the boundary of the state 

space at entry to the constrained subarc (i.e. f (t ) - a(t ) ^ 0) , it 

m e e 



follows that 
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^ L(t/) - L(t^ ) + P®(t^'^)){f(t/) - f(t )} 

B, “v e e ^ e. 

p (t ) = 

f (t ) - a(t^) 
me e 



where we have adopted the notation 



p (t^ ) = p (t^ ), 



and 



(95) 



Q (t ) = q (t ) . 

^ Q. ^ Q. 



When U* is discontinuous at entry to the boundary of the state space 

with X = a(t) , it follows from adapting the argument of Berkovitz [1] 
m 

on discontinuous "right-hand sides" to the usual development of the corner 
conditions (see [24]) 



and 



B/ “v B . , .m 

^ > + Pi 

q (t ) = q (t ) , 

^ e ^ e 

+ pa(t^). 

e e e 



(96) 

(97) 

(98) 



By (96) through (98) and the non-tangency assumption, it follows that 



L(t^)-L(t ) + (P®(t^))^{f(t^)-f(t )}+(Q^(t^))^{g(t^)-g(tj} 

B. —V e e^e /^e^e ^e ^^^e^e 

77^; 77 



where again we have used (95) . It should be noted that (99) reduces to 
(94) when is continuous at entry. We may also re-write (96) and (97) 

(or (91) and (92)) as 



p^t ) = P^t ^), 

e e 



B/ — . r\B/ +v 

q (t ) = Q (t ) , 

^ e ^ e 



and 



( 100 ) 
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The adjoint equations in the reduced state space may be obtained from (79) 

g 

and (80) by use of (78) to eliminate p (t) . From (78), we have 

m 



3f 



B 1 j3L ,-B,T 

Pm = - 3£- 137 > 37* - 



m 



8v 



since we have assumed ^ ^ along an optimal trajectory for (CPB) . 

“B B 

Since (t) is denoted as P (t) in the reduced state space, this 
becomes 



B 



3f 



Pm 3f Uv ^ 9v^ 

m 



( 101 ) 



8v 



Using (101) and denoting the adjoint variables in the reduced state space, 
(79) may be written as 

. . it _ 

^ 8x 



3f 


, B.T 


9g 


r8L 


/-B,T -* 


— 


(Q ) 


— 


1T“ + 

•■du 


(Q ) ^ 0- 

^ 3u'' X 


3x 




3x 








r3f ^ 







m 



3x 



+ 



(IL + (pB)T ^ 
l'3f ^3v ~ ’ 3v^ 



3f 



,3V , 



Similarly for (80) 



( 102 ) 



(q»)T . . it . !!: . (It + a 0* 

3y 8y ^ 3y ^3u 9u y 






m 



+ 



J 



m 

3v 



(l7 + (£,“)" ^ 



3f 



(103) 
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It should be noted that, to be precise, in the reduced state space we 
have 




X . for i=l, . . . ,m-l 
i?^k 




X . for i=l , . . . 
^ ±^k 



etc. However, to avoid excessive notation we will not be precise on this 
point and hope that the identification of the independent variables when 
partial derivatives are to be taken is clear from the context. When 



X 



m 



X 



m 



X 



m 



X 



m 



X (t) = a(t) for t ^ t ^ t with t < t . , 

m entry exit entry exit 



we have 



" f (t,x,x =ni(t) ,y,V (t,z)) = a(t) for t “ ^ t ^ ^ ’ 

m m ^ m ^ ^ entry exit 



—•k — 

except at manifolds of discontinuity of V (t,z) (i.e. except at the end 
points of the interval) . Differentiating the above identity with respect 
to X, we obtain 



which yields 



9f 



since 



m 



f 4- f V- = 0, 
m- m X 
X V ^ 



Sf 



m 



-a 

V- = - 

X 



3x 



3f ’ 
m 



(104) 



3v 



3v 



^ 0 along an optimal trajectory. Similarly we also obtain 



V- = - 



3f 



m 



f9f I 
m 

3v 



(105) 
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Using (104) and (105) , we may write (102) and (103) as 



•B T 

(P ) = 



It 

9x 



R T 

3x 






H 






3x 



. (ii + (pB)T f_ 

^3v ~ ^ 9v^ X ’ 



(106) 



<a“>" - - 17 - <t'>" I - <s'>" ^ ^ c; 






. (it + (pB)T f 

^3v ^ 3vJ y 



(107) 



CASE (2): y (t) = 3(t) for t ^ t ^ t with t ^ < t . 

^ T i entry exit entry exit 

We consider the optimal control problem (CPB) in a reduced state 

space in which the state variables are x. , . . . ,x ,y. ,...,y _ (also denoted 

1 ml n-1 

— T T — T 

as z = (x ,y )) and the corresponding adjoint variables are denoted as 
B B 

P , Q . Then the system dynamics are given by 



X = f (t,x,y,y =6(t) ,v) , 



n 






y = §.<t,x,^,y^=6(t),U (t.z)) 



with U (t,z) = U*(t,x,y,y =6(t)) being determined by g (t,x,y,y =3(t), 
^ ^^n n^^n 

— — • 

U (t,z)) = 3(t). We also will denote V(t,x,y,y =3(t)) as V = V(t,z). 

In the reduced state space we define a Hamiltonian as follows 



H^(t,z,P^,Q^,v) = L(t,x,y,y =3(t),U (t,z),v) 



r>>j /N/ 



+ (P^)^f (t,x,y,y =3(t) ,v) 

rs^ rs^ £l 



.b,t- 






+ (Q ) g(t,x,y,y =3(t),u (t,z)) - n. (t) {x -a (t) } , 
n 1 m 



( 108 ) 
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where 



n^(t) 



= 0 for X > a(t) , 
m 



^ 0 for X = a ( t) . 
m 



According to the results of Speyer [12], first order necessary conditions 
of optimality for (CPB) in CASE (2) are as follows: 



|l!. 0 or 0, 



(109) 









dL 


B.T 




3g 


dx 




(Q ) 


9x 


3L 


, B T 


T 


3i 




(P ) — - 


(Q ) 


— 


3y 


~ 9y 




8y 



'9L . ,^B,T 



9g 



9g 









( 111 ) 



Using (109), we may write (110) and (111) for > a(t) as 



•B T 

(p ) = 


_ IL _ 

9x 


(P®)^ 


9f 

9x 


(Q®)^ 


8x 


^9u 


(Q®) 


T 

9u^ 


-* 

u 

X 












- \ 


[|^ + 
^ dV 


(P®)^ 


9f 


"x ’ 




(112) 


•B T 

(Q ) = 


_ IL _ 




9f 


(Q®)^ 


9g 


If-* 

^ dU 


«“) 




_:Sc 

u- 

\7 






3y 




9y 




9y 






y 












- ] 






9f 

f) 


V- . 

y 




(113) 



“B B B 

H , P (t) , and Q (t) are continuous except at manifolds of 
discontinuity of U* and possibly at junctures to the boundary of the 



state space with y^ = 3(t). Thus, we also must consider necessary condi- 
tions for a juncture between an unconstrained subarc and a constrained 
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subarc on the boundary. The results of Speyer and Bryson [17] tell us 
that 



and 



B . +v B. -V 
p (t ) = p (t ) , 
0 0 



Q (t ) = q (t ) , 
0 0 



(114) 



where T = t and t denotes a left hand limit. It also may be 

e entry e 

shown that 



= H^(t ) - q®(t )B(t ). 
e e n e e 



(115) 



This may be written as 



L(t ,x,y,y =e(t ) ,U (t^,z) ,v*(t^) ) + (P® (t^) )^f (t ,x,y ,y =g (t ) ,v*(t^) ) 
0 <^^<^^n e e^ e ^ 0 *^ 0 ^A^n e e 



+ = L(t ,x,y,y =B(t ) ,U (t ,z),v*(t )) 

^e^e^^n e e^ e^^n e e^ e 



+ (P^(t ))^f(t ,x,y,y =B(t ),v*(t )) + (q® (t ) )\ (t ,x ,y ,y =B ( t ) ,U* (t , z) ) 
^e/^^e^^n e e ^ 0 ^e^<^n e 



-q^(t )e(t ) , 
n e e 



(116) 



or (using (114)) 






>v*(t^)) - L(t^.x,£,y^=e(t^) ,U*(t;,i) .vM^)) 



+ (P^(t^))'^{f (t ,x,y,y =B(t ) ,v*(t^)) - f (t ,x,^,y =g (t ) ,v* (t ) ) } + 
^e ^ 0 ^A^n e e ^ 0 <>*^^n e e 



(Q®(t^))^{g(t ,x,y,y =g(t ) ,U*(t^,z)) - g(t ,x,y,y =B(t ) ,U*(t ,^))}. 
^ e <^e^^ n e e^ e^<^n e e^ 



( 117 ) 
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When the state trajectory does not enter the boundary of the state space 

—Vc — — • 

tangentially, we have that g (t ,x,y,y =3(t),U (t ,z)) - 3(t ) 0, so 

ne^^n e 

B “ 

that (117) may be solved (uniquely) for * Then 



L(t^)-L(t )+(P^(t^))'^{f(t^)-f(t^)}+(Q®(t^))^{g(t^)-g(t^)} 
D , —V e e^e ^e e ^e x^e^^e 






g (t ) - e(t ) 
n e e 



(118) 



When g (t ) - 3(t ) ^ 0 and v*(t ) (i.e. no manifold of discontinuity 

nee e 

of at entry to reduced state space) , (118) becomes 



^ L(t'^) - L(t") + (Q®(t‘^))hg(t'^) - g(t")> 

q (t ) = . 



— — — • 

When g (t ,x,y,y =3(t ) ,U (t ,z)) - 3(t ) = 0, then the juncture condi- 
ne'^x.n e e^^ e 

B ~ 

tions (114) and (115) (I.e. (117)) do not determine q (t ) . 

n e 

To summarize, we have for (CPB) 



for ^ g(t)) ^ 3(t) almost everywhere in time ; 







( 120 ) 



( 121 ) 
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for y (t) = 3(t) for t ^ ^ t ^ t with t ^ < t 

^ entry exit entry exit 

and X (t) > a(t) almost everywhere in time: 

m ^ 






• B T 

(Q ) = 



3L _ 




9£ 


/^B, T 




9x 


(P ) 


9x 


(Q ) 


9x 










r9L 








“ 1 


Uv 


9L 


/,.B,T 


9f 


/^B,T 


9g 




(P ) 


_ 


(Q ) 


— 


9y 




9y 




9y 








_ j 


r9L 








1 


^9v 



- ( 1 ^ * 0 * 

^dU dU^ X 



V* 

- ’ 9vJ X 



^3u 



.B,T 



9g 



-■k 



3u^ y 



R T * 

^ t- 

^ 3v^ y 



for X (t) = a(t) for t ^ 
entry 



X XX 

m . , m ^ ^ m 

t t . with t < t . 

exit entry exit 



and 3(l) almost everywhere in time : 



•B T 
(P ) = 



ik 

3x 






3f 



3x 



9g 



(Q*^)^ J|il+ (Q^)^ % U- 

^ -- ^9u ~ 9u^ X 

9x ~ 



9f 



- It ^ (L”)‘ £) t , 



3v“^ X 



• B T 

(Q ) = 



9L 

9y 



R T 

a 7 



I - It - <a“)" ^ 



-* 
J U 

du*’ y 



. (it + (pB)T t} V* 

**3v ^ 9v** y 



(122) 



(123) 



(124) 



( 125 ) 
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5.3. Equivalence of Multipliers in the Two Problems . 

By (70) through (75) and (120) through (125), we see that p^, 

A B B 

q and p , q satisfy the same differential equations (appropriately 
modified on the boundary of the state space) . In order to complete the 
proof of the equivalence of these multipliers it remains to show that 
they take on the same terminal values and possess the same continuity 
properties (both interior to and on the boundary of the state space) . 

Thus, it remains to discuss 

(1) terminal values of the adjoint variables, 

(2) continuity of the adjoint variables across manifolds of discon- 
tinuity of strategies at interior point of state space, 

(3) juncture conditions at boundary of state space. 

5.3.1. Terminal Values of the Adjoint Variables . 

For (CPA), there are several distinct cases that must be considered. 



CASE (Al) : X (T) > a(T), y (T) < B(T). 

m — ^ 



TT- 



From (31) , (32) , and (33) we have 



(P^(T))'^ = 



3x(T) 



+ a 



A 3F 



(q^(T))^ = 



3y(T) 



+ 0 



9x(T) ’ 

A 3F 
9y(T) ’ 



L + (p^(T))^f + (q^(T))^g + H + II 



= 0 . 



We may write the above as 
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,T * 



” -I 

'Mn 


0 


OF n 




r P^(T) ' 




r 1 


Ox(T) 






Ox(T) 






T 








T 


0 


-I 


9F^ 




q (T) 




SG 


o;!^(t) 






Oy(T) 




T 


11 




A 




3G 


f 


£ 


OT J 




_ a 




- 3T -1 



(126) 



where denotes an m x m identity matrix. It should be recalled that 

we have assumed that the optimal path is not tangent to the terminal mani- 
fold. This non-tangency assumption (see [3] or [15]) yields that there is 
a unique solution to equations (126) . 

X 

CASE (A2) : x (T) = a(T) with x (t) = a(t) for t ^ ^ t ^ T with 

i ^ ^ entry 

t ™ < T and y (T) < 3(T) . 

-^ntry = — ■ 

In this case we have 



(p^(T))^ = -3^ + -4^ 



3x(T) 



3x(T) 






3^(T) 



3y(T) ’ 



L + (P^(T))^f + (/(T))^g + If + H = 0- 






We may write the above as 



y 


0 


3F -1 




i 

H 

1 




r 1 


in-l 


Ox(T) 






3x(T) 


0 


-I 

-Ml 


~ T 
3F 




Q^(T) 




Tg^ 


3^(T) 






- 3^(T) 


1 


T 

g 


1£ 

3T 




-A 
_ 0 




- L 

^ 3T J 



-1 t=T 



( 127 ) 
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Again, the non-tengency assumption yields that there is a unique solution 
to equations (127). 

It should be pointed out that there are two ways in which we may 
think of equations (127) as arising: 

(a) the usual transversality conditions in a reduced state space in 

which the state variables are x- , . . . ,x _ ,y. , . . . ,y , 

1 m-1 1 n 

(b) the usual transversality conditions in the original state space. 

Let us now show that (a) and (b) are equivalent. By (31) through (33), 

we have in the original state space in which the state variables are 

X , . . . ,x y, . . . ,y that 
1 m n 



(p^(T))'^ = 


3G . A 


3F 


3x(T) 


3x(T) 


/ A. , T 

(q (T)) = 


3G , A 


3F 


ay(T) ° 


9y(T) 


H^(T) + 


- + v'^a(T) = 


■■ 0. 



(128) 

(129) 

(130) 



We may write the last equation as 



A T A T A* 

L + (p (T)) f + ( 3 ^'^(T))^g^ + v^a(T) = 0. 



From (27) and (128) we have 



Thus 



^1 3x (T) ~ 
m 



L ^ (/(I))h + (/(T))\ + 11^ + - p*(T)li(T) - 0, 

m 



It is convenient to write the above as 



L+ (£^T))"f + p>{f^-<i(T)} = (gA(T))Tg^+|i^ + ^a(T) 

m 



= 0. (131) 
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X 

Let us recall that x = ot(t) for t ^ ^ t ^ T. Thus, we have 

m entry ’ 



dx 



m 






= f (t,x,x =a(t) ,y,V”(t,z)) = a(t) for t < t ^ T, 



dt m m 



entry 



or 



f - a(T) = 0, 
m 



(132) 



Using (132), we then obtain from (131) that 



L+ (£V))^f + (T) = 0. 

m 



(133) 



Recalling that the state variables in the reduced state space are , . . . , 



K (or X, y), we have 

m— 1 1 n ^ 



“ Ut ^ (Sx - * dt 

x(T),y(T) x(T),y(T) T,x(T),y(T) 



m 



(T) , 






since on the constrained subarc we have x = x (t) . Hence, in the reduced 

m m 

state space (133) becomes 



L+ (AT))"f + (At))\+ 



= 0 , 



x(T) ,;/(!) 



or 



L + (£\t»"£ + . (If). 



^ A^aF^i 



^^'x(T),;j;^(T) x(T) ,;^(T) 



= 0. (134) 



Making the identifications P^(T) = p^(T) , Q^(T) = q^(T) , and , 

rw 



we see that (127) follows from (128), (129), and (134). 



A-40 



CASE (A3): x (T) = a(T) but x (t) > a(t) for t 6 (T-6,T) and 

-m m — 

y^^(T) < 3(T) . 

In this case we essentially have an additional terminal constraint 

X (T) = a(T) (and the condition that x (t) > a(t) for t G (T-6,T) 
m m 

where 6 > 0) . From (31) , (32) , and (33) we have 



, dG . A 9F A,,m^T, 



, 3G , A 9F 

(q (T)) = . + O 



9y(T) 



9y(T) ’ 



L + (p (T)) £ + (q (T)) + — + a — + v^a(T) = 0. 



We may write the above as 

.T 



-1 



0 



-1 



3F 



3x(T) 
.T 



3F 



9y(T) 

9T 





r A,„, n 
P (T) 






A,„v 

a (T) 


II 




A 

_ a _ 





9G . A,,m^T- 

8;^ + ^ 



9G 



- L - 



3i(T) 



9G A* 



(135) 



t=T 



The non-tangency assumption implies that equations (135) have a unique 
solution which we denote as 



p^(T) = R^(v^) , 
q*(T) - S*(V*) , 
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AAA 

Hence, the terminal multipliers p (T) , q (T) , a depend (continuously) 
on a single parameter, v^. The value of this parameter, however, cannot 
be determined from the transversality conditions but must be determined 
by considerations "in the large" (see Section 7.3 of [5]). The parameter's 
value is chosen so that the system is steered to the given terminal state 
(see [25] for corresponding one-sided (optimal control) problem). Thus 

A 

V. is chosen so that x (T) = a(T) but x (t) > a(t) for t 6 (T-6,T). 

1 mm 

OTHER CASES . These are treated analogously to the above. 

For (CPB) , there are again several distinct cases that must be 
considered. It should be recalled (see Section 4 above) that the optimal 
path is not tangent to the terminal manifold. 



(Bl): X (T) 


> a(T), 


.yAJ) 


< 


e(T) . 






m 

Similar to 


case for 


n 

(CPA), 


, we find that 






-I 


0 


T 

3F -n 




1 

H 

1 




T 

r 3G -| 




3x(T) 






3x(T) 




0 


-I 


T 

3F^ 




q^(T) 




T 

3G 




3y(T) 






9y(T) 






T 

g 


3F 

3T 


1 


B 

a _ 




- L 

3T _ 



(136) 



-i t=T 



Again, the non-tangency assumption yields that (116) has a unique solution. 

By (126) and (136) we see that p^(T) , q^(T) , and p^(T) , 

B B 

q (T) , a satisfy the same set of equations which have a unique solution. 
Hence, in this case we have shown that 

P^(T) = p^(T) , 
q^(T) = q®(T), 

= a®. (137) 



A-42 



X 

CASE (B2) : x (T) = a(T) with x (t) = a(t) for t ^ t ^ T with 

^ entry 

t ^ < T and y (T) < 3(T). 

-entry — ^ - n - — ' 

Similar to case for (CPA) , we find that in the reduced state space 



^-1 

0 




0 


3F - 




I 

1 




3G ~ 


3x(T) 






3x(T) 


-I 

'Ml 


T 

3F 




Q®(T) 




T 

3G 


3y(T) 






3y(T) 


T 

g 


3T - 




-B 

a 




- L - ^ 

- ^ 3T J 



(138) 



Again, the non-tangency assumption in the reduced state space yields that 
(138) has a unique solution. 

By (127) and (138) we see that P^(T) , Q^(T) , and P^(T) , 

B “B 

Q (T) , a satisfy the same set of equations which have a unique solution. 
Hence, in this case we have shown that 

P^(T) = P^(T) , 

Q^(T) = Q^(T), 

= a^. (139) 



CASE (B3): x (T) = .g(T) but x (t) > g(t) for t € (T-6 ,T) and 

y (T) < g(T) . 

In the necessary conditions of optimality for (CPB) , we have ^ 0. 
Similar to case for (CPA), we find that 



-I 


0 


3F ^ 




1 

/-N 

H 

1 




3x(T) 






0 


-I 

'Ml 


T 

3F 




q®(T) 




3y(T) 








T 

g 


iZ 

3T 




B 

_ a __ 





3 G B / ^ m^ T ““ 



9G 



3y(T) 



IG 

9T 






( 140 ) 
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Again, the non-tang ency assumption yields that (140) has a unique solution 
which we denote as 

p^T) = R®(v®), 

q (t) = S (v ) , 

B vB. B, 

^ = I (v^). 

g 

As was the case for (CPA) , the value of cannot be determined by the 

transversality conditions at t = T but is determined by considerations 
"in the large." 

In this third case, the problem essentially behaves as though it 
had two terminal equality constraints. With more than one terminal equality 
constraint, our results apply to differential games for which 

p^(T) = p®(T), 

/Vi/ /V/ 

q^(T) = q®(T), 

/V/ /V/ 

^ ^ /I / 1 ^ 

a = a . (141) 

This type of qualification was apparently first noted by Schmitendorf and 
Citron [16]. We have not been able to prove (141) for differential games 
with p > 1 (where p = the number of terminal equality constraints) (nor 
has anybody else apparently). However, it seems plausible that (141) holds. 

Let us elaborate about the plausibility of (141) . In (CPA) we have 
V*(t,x,y) given, and we determine u*(t) = U*(t,x,y); while in (CPB) we 

/V/ /V/ 

have U*(t,x,y) given, and we determine v*(t) = V*(t,x,y). Thus, in 

/V/ /V/ 

both problems we have the same optimal strategies U*(t,x,^) and • 

T T 

For a given initial point find the same optimal trajectory 
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(x* (t) ,y* (t) ) for 0 ^ t ^ T in both problems. Hence, we would expect 
to be able to find the same multipliers (they may not be unique) in both 
problems, since we have the same adjoint equations, transversality conditions 

(as well as system dynamics), and optimal trajectories in both problems. 

A, . A , B . ^ , 

p (T) , q (T) , a and p (T) , q (T) , a satisfy the same equations 

(135) and (140) (provided that we equate with vh . If p^(T) , q^(T) , 

g 

a is not unique, we will agree to pick it so that (141) holds. 

OTHER CASES . These are treated analogously to the above. 

Thus, we have shown that (137) holds in CASE (1) and (139) in CASE 
(2) . In cases like CASE (3) , our results apply to differential games in 
which (141) (or the equivalent) holds. 

5.3.2. Continuity of the Adjoint Variables Across Manifolds of Discontinuity 
of Strategies at Interior Point of State Space . 

Let M(t,x,y) = 0 define the manifold of discontinuity and t, 

rs^ Q 

denote the time at which this manifold is reached along a particular 
trajectory. Assuming that the optimal path is not tangent to any manifold 
of discontinuity at an interior point of the state space, standard argu- 
ments (see [3] or [15]) in which the manifold of discontinuity M plays 
the role of the terminal manifold then yield 




k, B,-. 

g. (t.) = q (,). 



and 



(142) 



A-A3 



Again, the non-tangency assumption yields that (140) has a unique solution 
which we denote as 



P^(T) 



q (t) 



^ 1 

„B, B, 

S (v ), 
^ 1 



rB f Bv 

i c^i) 



As was the case for (CPA) , the value of cannot be determined by the 

transversality conditions at t = T but is determined by considerations 
"in the large." 

In this third case, the problem essentially behaves as though it 
had two terminal equality constraints. With more than one terminal equality 
constraint, our results apply to differential games for which 



p (T) = p (T), 
q^(T) = q®(T), 

A B /-, / -1 \ 

a = a . (141) 



This type of qualification was apparently first noted by Schmitendorf and 

Citron [16] . We have not been able to prove (I4l) for differential games 

with p > 1 (where p = the number of terminal equality constraints) (nor 

has anybody else apparently). However, it seems plausible that (141) holds. 

Let us elaborate about the plausibility of (141) . In (CPA) we have 

V’*^(t,x,y) given, and we determine u*(t) = U*(t,x,y); while in (CPB) we 
0^ 0^ 0^ 

have U*(t,x,y) given, and we determine (t) = V*(t,x,y). Thus, in 

both problems we have the same optimal strategies U*(t,x,j^) and 

T T 

For a given initial point find the same optimal trajectory 
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(x* ( t) ,y* (t) ) for 0 t :S T in both problems. Hence, we would expect 

<rw 

to be able to find the same multipliers (they may not be unique) in both 
problems, since we have the same adjoint equations, transversality conditions 

(as well as system dynamics), and optimal trajectories in both problems. 

AAA B B B 

P (T) , q (T) , a and p (T) , q (T) , a satisfy the same equations 

(135) and (140) (provided that we equate with P^(T) , q^(T), 

g 

a is not unique, we will agree to pick it so that (141) holds. 

OTHER GASES . These are treated analogously to the above. 

Thus, we have shown that (137) holds in CASE (1) and (139) in CASE 
(2) . In cases like CASE (3) , our results apply to differential games in 
which (141) (or the equivalent) holds. 

5.3.2. Continuity of the Adjoint Variables Across Manifolds of Discontinuity 

of Strategies at Interior Point of State Space . 

Let M(t,x,y) = 0 define the manifold of discontinuity and t , 

^ d 

denote the time at which this manifold is reached along a particular 
trajectory. Assuming that the optimal path is not tangent to any manifold 
of discontinuity at an interior point of the state space, standard argu- 
ments (see [3] or [15]) in which the manifold of discontinuity M plays 
the role of the terminal manifold then yield 

A/ -X B. 



A, B--. 

g. (t.) = g. (.)• 



and 



(142) 
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If M is a manifold of discontinuity for only U* , then in (CPA) 
the usual corner conditions at an interior point of the state space yield 



and 



A/“v A, 

P = R. 



A, A, 

q (t ) = q (t ) 

^ Q ^ a 



(143) 



Similarly, if M is a manifold of discontinuity for only V*, then in 
(CPB) the usual corner conditions at an interior point of the state space 
yield 



and 



B > — V B , H-. 

Z = £ (t ), 



^ ^ / "In 



(144) 



If M is a manifold of discontinuity of both U* and , then 

we have, for example, in (CPA) that [1] for all differentials dt,, dx, 

d ^ 

dy along the manifold 

{H^(t^)-H'^(t^) }dt^ - {p^(t^)-£^(t^) }^dx - (£^(tj)“£^(tj) = 0. (145) 



5.3.3. Juncture Conditions at Boundary of State Space . 

Let us consider the juncture conditions which must hold between the 



adjoint variables at the entrance to a constrained subarc on which x (t) = 

XX XX “ 

/V _ p m m T . - m m 

«(t) for t 6 [t ^ ] with t ^ < t . , 

entry exit „ entry exit 

A 

similarly treated. We denote t ^ as t . 

entry e 



Other cases are 



For (CPA) we have shown that 



—A. _A. +v 

Z = I > 
q"^(t ) = Q^(t^) , 



(146) 



( 147 ) 
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and 



H^(t ) - p\t )a(t ) = 

e m e e e 



( 148 ) 



Assuming that the optimal trajectory does not enter the boundary of the 

state space tangentially, (146), (147), and (148) yield a unique value 

for p^(t ) . 
m e 



, L(th - L(t ) + (p"^(th)hf(th - f(0> + (Q^(th)hg(th - g(t >} 

p (t ) = 

me - * 



f (t ) 
m e 



ot(t ) 
e 



When there is no manifold of discontinuity of U* at entry to the reduced 
state space, the above further simplifies to 



L(th - L(t ) + (P^(t^))^{f(t^) - f(t )} 

p (t ) . 

® S r /~\ * /^ \ 

) - a(t ) 
me e 



(150) 



For (CPB) we have shown that (for a first order SVIC) assuming that 
the optimal trajectory is not tangent to the boundary of the state space 
at a juncture point 



-B 

P 






(151) 



L(th - L(t ) + (p®(th)hf(t^)-f(th) + (Q®(th)hg(th-g(0} 

D , “v 0 0 *^0 *^ 0^0 ^0 ^ 0^0 

P„(t„) = : : 



f (t ) 
m 0 



a(t ) 
e 



(i; 



and 

q®(tj=Q®(th. (153) 

When there is no manifold of discontinuity of U* at entry to the reduced 
state space, equation (152) further simplifies to 
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_ L(t^) - L(t ) + (P®(t'^))'^ fCt”^) - f(t ) 

^ _ B e e ^ e ^ e 

^ e * 



f (t ) - a(t ) 
me e 



(154) 



H" 

In the original state space we have (recall that we denote p (t ) as 

^ e 

B *4" 

P (t ), etc., in the reduced state space) when U* is continuous at 
e 

entry 



B 

£ 






B / +v 

■ £ 



> 



and 



B 

q 



(t„) = q^(t^) • 

e ^ e 



(155) 



The above yield that when ^^(t) = £^^(t) and q^(t) = q^(t) for 
t^ < t ^ T, it follows that provided the optimal trajectory is not tangent 
to the boundary of the state space at a juncture point 



A, B. 

P (t ) = p (t ) , 

^ e ^ e 

and (156) 

A, B, 

q (t ) = q (t ). 
e ^ e 



A B 

Equivalence of the Adjoint Variables ^ (t) and ^ (t) . 

It is convenient to denote 

(X^(t))^ = ((p^(t))^,(q^(t))^), (157) 

B A B 

and similarly for X (t) . The equivalence of X (t) and X (t) follows 

from the facts that 

A B 

(1) on the terminal manifold we have X (T) = X (T) , 

(2) as we work backwards from the terminal manifold to the first 

manifold of discontinuity, both XA(t) and XB(t) satisfy the 
same differential equation so that" XA(t) = X^(t) for t^< t ^ T 
where t^ denotes the time closest to T such that either U* 
or is discontinuous, 
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(3) working backwards across the first manifold of discontinuity, 
we have X (t^) = X (t^), 

(4) in this manner one works backwards agross each manifold of dis- 
continuity and shows that X (t) = X (t) for 0 t ^ T, 

Thus, (70) through (75), (120) through (125), (37), (139), (141), (142), 

and (156) yield that 



for 0 :S t :S T: 



A, X B, , 

p (t) = p (t) , 



and 

A, . B, , 
(t) = (t) . 



(158) 



6 . A Problem with More General Dynamics . 

The above development also applies to the following problem (denoted 
as ’’Problem II”) with slightly more general system dynamics. 



Problem II. 



maximize minimize {G(T ,w(T) ,x(T) ,y (T) ) + 
U V "" 



L(t ,w,x,y,u,v)dt} , 



0 



subject to: 



w = f (t,w,x,y,v) , 

X = g(t,w,x,y,u,v) , 
y = h(t,w,x,y,u) , 



u(t), v(t) 
w(t) ^ a(t) 
y(t) :S 8(t) 



are unrestricted (scalar) 
for all t e [0,T] 
for all t € [0,T] 



strategic variables, 

(scalar state variable 
inequality constraints) , 



with scalar terminal condition F(T,w(T) ,x(T) ,y (T) ) = 0, 



Considering the results of Sections 5.3.2 and 5.3.3, we see that the subse- 
quent statement applies whether the manifold of discontinuity lies interior 
to the state space or on the boundary. 
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where we assume that all functions are smooth enough to insure the existence 
of all partial derivatives required. In Problem II above, x is an n- 
vector of state variables. 

It will be convenient to denote the vector of state variables as 
T T 

z = (w,x ,y) = (w,x ,x , ...,x ,y). 

^ 1 z n 

We will then sometimes write terms in a more convenient form as 



L(t,z,u,v) = L(t ,w,x,y,u,v) . 



Before we state necessary conditions of optimality for Problem II, 



let us state the assumptions that we make for Problem II: 



(1) the differential game has value and a saddle point in pure 
strategies , 

(2) the solution to Problem II is normal (see [ 1 ]), 

(3) the optimal path is not tangent to the terminal manifold or to 
any manifold of discontinuity of U* or V* or to the boundary 
of the state space at entry to a constrained subarc which lies 
on the boundary, 

(4) to insure the existence of value for the differential game we 
assume** 



(a) L(t,z,y,v) = L (t,z,u) + L^(t,z,v), 

(b) g(t,z,u,v) = g (t,z,u) + g (t,z,v), 



(5) the first time derivative of y(t) explicitly contains the strategic 

variable u and that (y) = h (t,w,x,y,u) / 0 along an optimal 

u u ^ 

trajectory; we further assume that (y) = 0, i.e. h does not 

depend on v at all, 



The development of these necessary conditions is similar to that for 
Problem I. 

As noted above, several other technical conditions must be satisfied to 
guarantee the existence of value and of a saddle point in pure strategies 
to Problem II (see Chapter 6 of [9]). 
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(6) the first time derivative of w(t) explicitly contains the 

strategic variable v and that (w) = f (t,w,x,y,v) f 0 along 

an optimal trajectory; we further assume Yhat Tw) = 0, i.e. 
f does not depend on u at all. 

Now we define 



H(t,z,P,n,u,v) 

rw 



L(t,z,u,v) + Xf(t,z,v) + p g(t,z,u,v) 






where 



and 



and 



+ qh(t,z,u) - n, (t){w-a(t) } - (t) {y~6(t) } , 

z 






H2(t) 



= 0 
^ 0 

= 0 
^ 0 



for w < a (t) , 
for w = a (t) , 

for y < B(t) , 
for y = e(t) , 



(159) 



$(T,z(T),o) = G(T,w(T),x(T),y(T)) + aF(T,w(T) ,x(T) ,y(T)) . (160) 

0 ^ 



In order that the strategy pair (U*,V*) be a saddle point of the criterion 
functional it is necessary that there exist unique functions X(t), p(t), 

q(t) , q^(t), and ^ 2 ^^^ constants a, and such that the 

following conditions hold 



3u 



0 or 





or 




-X = H = 
w 



3L 



3f 



3w ^ 3w 




3v 



T 3g 
3w 



P 

0 ^ 





(161) 


3g 


(162) 


3h 

3w '^1’ 


(163) 



In other words, we assume that w(t) ^ a(t) and 
appropriate types of first order SVIC*s. 



y(t) ^ 3(t) are the 
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•T „ 9L , ,9f T , 3h 

r>>^ 

• - U - 9L ^ , 9f ^ T ^ 9h 

y 9y 9y ~ 9y 9y ^2’ 



(164) 

(165) 



9$ 



p'^(T) = 



9x(T) ’ 



. = 0 for w(T) > a (T) , 

^ ^ 0 for w(T) = a(T) ,* 



(166) 

(167) 



9 $ 

■ MT ) - '’2 "2 



= 0 for y(T) < g(T), 
k 0 for y(T) = B(T) , 



H(T) + H - v^a(T) - v^ad) = 0, 



(168) 

(169) 



Analogous to the Weierstrass condition, we also have the max-min principle 



H(u,v*) ^ H(u*,v*) ^ H(u*,v), 



(170) 



which holds for all admissible u and v. H, X(t), ^ q(t) 

are continuous functions except at manifolds of discontinuity of both U* 
and V*, where for all differentials dt, dw, dx, dy along the manifold 



)dt - )dw - )^dx - (q'^-q )dy = 0. 



(171) 



In other words, at both entrances to and exits from constrained 
subarcs on the boundary of the state space , we have 



and 



X(t ) = X(t^), g^(t ) = £(t^), q(t ) = q(t^), 



H(t^) = H(t^), 



(172) 



(173) 



It should be recalled that this statement must be modified for an absorbing 
state boundary. 
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except possibly at manifolds of discontinuity of both and V*, 

where all the dual variables are continuous except those corresponding 
to w or y which describes the boundary. 

The above first order necessary conditions of optimality for 
Problem II are analogous to the optimal control theory results of Jacobson, 
Lele, and Speyer [12]. These results may also be written in a form 
analogous to that of optimal control theory results of Berkovitz [2] and 
Gamkrelidze (see Chapter VI in [14]). This is done by introducing new 
multipliers X(t) such that 



X(t) = X^(t) - y^(t), (174) 

p(t) = X.(t), (175) 

q(t) = ^^(t) - (176) 

It is well known that we may then make the following identifications 

n^(t) = - for i = 1,2. (177) 



7 . A More General Type of State Variable Inequality Constraint . 

It is of considerable interest and import to consider a more general 
type of SVIC. We, therefore, consider the following problem (denoted as 
"Problem III") . 
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Problem III. 

T 

{G(T,x(T) ,y(T)) + L ( t ,x ,y ,u ,v)dt } , 

0 

# 

X = f (t,x,y,v) , 

<N./ 

y = g(t,x,y,u) , 

u(t) , v(t) are unrestricted (scalar) strategic variables, 

C_ (t,x) ^ 0 for all t G [0,T] (scalar state variable 

inequality constraints) , 

C„(t,y) ^ 0 for all t € [0,T] 
z ^ 

with scalar terminal condition F (T ,x(T) ,^(T) ) = 0, 



maximize minimize 
U V 

subject to: 



where we assume that all functions are smooth enough to insure the existence 
of all partial derivatives required. In Problem III above, x is an m~ 
vector of state variables, and y is an n-vector of state variables. 

It will again be convenient to adopt the notational conventions (6) and 
(7). 



We again make the assumptions (1) through (3) given in Section 6 
above and modify (4) through (6) as follows: 



(4) to insure the existence of value for the differential game we 

assume L(t,z,u,v) = L-(t,z,u) + L^(t,z,v), 

^ J. ^ z 

(5) the first time derivative of C 2 (t,^(t)) explicitly contains the 
strategic variable u and that (C 2 )^(t ,x,y ,u(t) ) ^ 0 ^glong^gn 



optimal trajectory; we further assume that (^2^v ” ^ 
= 0, i.e. does not depend on v at all. 



_2 + _2 
3t 3y 



(6) the first time derivative of CT(t,x(t)) explicitly contains 

1 

the strategic variable v and that (C^ ) (t,x,y,v(t)) ^ 0 along 

1 V 



an optimal trajectory; we further assume that (C, ) = 

3C^ ^ 

^ “0, i.e. does not depend on u at all. 



3u 



3t 



3x 
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Unfortunately, it is not convenient to develop necessary conditions 
for Problem III in the same manner in which they were developed for 
Problem I. We will now state necessary conditions of optimality for 
Problem III. Although a proof has been developed for these conditions, 
it will be given elsewhere in the future, since it is rather lengthy. 

First we define 



H(t,z,p,q,n ,u,v) = L(t,z,u,v) + p f(t,x,y,v) 









+ q g(t,x,y,u) - rii(t)C (t,x) - n (t)C (t,y), (178) 



where for i = 1,2 



n^Ct) 



= 0 for (-1) ^ 0, 



^ 0 for C . = 0, 
1 ’ 



and 



<D(T,z(T),a) = G(T,x(T),y(T)) + oF(T,x(T) ,y(T) ) 



(179) 



In order that the strategy pair (U^,V*) be a saddle point of the 

criterion functional it is necessary that there exist unique functions 

p(t)> q(t), Hi (t) , and rio(t) and constants v. , , and a such 

0^ i, Z 1 z 

that the following conditions hold 



3H - 3L , T . 

— = 0 or — +q — =0, 

8u 8u ^ du 

3H - 3L , T „ 

— = 0 or — + p — = 0, 

3v 3v ~ 3v 



•T 




T 


3f 

0S^ 


T 


3Ci 


P 


TT . 

= H = -7 — + 
X 9x 


P 

0>0 


3x 

0^ 


~ 3x 
0>0 


^1 3x ’ 
0^ 


•T 


3L . 


T 


3f 


T 


30^ 


q 

0^ 


It 

u 

it 


P 

0S^ 


37 

0^ 


~ 3y 
0>0 


'^2 3y ’ 



(180) 

(181) 

(182) 



( 183 ) 
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T 

P (T) 



q^(T) 



3<t' 

9x(T) 



9y(T) 



9Ci j 




0 


for 


C (T,x(T)) > 0, 
1 ^ 


'’l 3x(T) ”l 


1 












0 


for 


C. (T,x(T)) = 0, 
1 ^ 


9 C 2 1 




0 


for 


C,(T,y(T)) < 0, 
z ^ 


“2 3y(T) “2 ] 


u 


0 


for 


C„(T,y(T)) = 0, 



(184) 



(185) 



8C 

^ H + "1 IF 



+ V, 



2 IF 



= 0 , 



(186) 



Analogous to the Weierstrass condition, we also have the max-min principle 
(18). H, p(t), and q(t) are continuous functions except at manifolds 
of discontinuity of both U* and V*, where for all differentials dt, 
dx, dy along the manifold (19) holds. 

The above first order necessary conditions of optimality for Problem 
III are analogous to the optimal control theory results of Jacobson, Lele, 
and Speyer. These results may also be written in a form analogous to that 
of optimal control theory results of Berkovitz and Gamkrelidze. This is 
done by introducing new multipliers X(t) such that 

T T 

p (t) = (t) - (t) -r— , 

1 d X 



q^(t) = xht) - M„(t) 




(187) 



It is well known that we may then make the identifications (25) . 

As should be clear to the reader, Problem I is a special case of 
Problem III. However, the development of necessary conditions of optimality 
for Problem III is much more tedious and not the same as for Problem I. 

The development of these more general conditions will be given in the 
future . 

It should again be recalled that this statement must be modified for an 
absorbing state boundary. 
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8 . Further Extensions . 

There are two directions of import in which the above results may 
be extended : 

(a) additional inequality constraints on the strategies, 

(b) higher order SVIC’s. 

It is of interest to consider a problem (denoted as “Problem IV") 
similar to Problem or Problem II in which there are additional inequality 
constraints on the (vector) strategies U and V. 



Problem IV. 



maximize minimize {G(T,w(T) ,x(T) ,y(T)) 
U V 




L(t,w,x,y,u,v)dt}, 



subject to: 



w = f (t,w,x,y,v) , 



X = g(t,w,x,y,u,v) , 



y = h(t,w,x,y,u) , 



u(t) is p-vector of strategic variables which satisfies 
u(t) 6 fi^(t,w,x,y) for all t 6 [0,T], 



v(t) is q-vector of strategi variables which satisfies 
v(t) 6 5^^(t,w,x,y) for all t 6 [0,T] , 



w(t) ^ ot(t) for all t € [0,T] 
y(t) ^ B(t) for all t € [0,T] 



(scalar state variable 
inequality constraints) , 



with scalar terminal condition F(T,w(T) ,x(T) ,y(T) ) = 0, 



a 



u 



{u K(t,w,x,y,u) ^ 0} , 



where 
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and 

= {v I R(t,w,x,y,v) ^ 0}. 

It is a straightforward matter to combine the previous development 
of Berkovitz [3] with our development for Problem I to develop necessary 
conditions of optimality for Problem IV. Our applications of the results 
of this appendix will be to Lanchester-type differential games of this 
form (see equations (1) above) . 

Another extension of import is the extension of our results on 
differential games with SVIC’s to problems with higher order SVlC’s. 
Similar one-sided optimization problems have arisen in our study of 
Lanchester-type optimal control problems [26] (see also [23]). Therefore, 
we consider the following problem (denoted as "Problem V") . 



Problem V. 

maximize minimize {G (T ,x (T) ,y (T) ) 
U V 




L(t,x,y ,u,v)dt} , 

/N./ 



subject to: 



X = f (t,x ,x ) , 



l2 = 



where 



u(t) , v(t) are unrestricted (scalar) strategic variables, 

C^(t, 3 i^) ^ 0 for all t 6 [0,T] state variable 

^2^^’^1^ ^ ^ ^ ^ [OjT] inequality constraints), 

with scalar terminal condition F(T ,x(T) ,y (T) ) = 0, 



T 

X 



T T T 



, T T. 
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( 1 ) 



We make the following assumptions about C^(t,x-) 

k k k 

(C^)^ ~ 0 for k = l,...,p~l where (C^) = — 

dt 



in Problem V: 



> 



(2) the first derivative of C^(t,x^(t)) which explicitly contains 

the strategic variable v in the p-^ and ^ ^ ^ along 

an optimal trajectory, 

k 

(3) ^ ^ ^ (i.e. for all integers k ^ 1) • 



Similar assumptions are made about C (t,y ). Other assumptions that are 
required in the development of necessary conditions are similar to those 
made in previous sections. The results of Jacobson, Lele, and Speyer 
[12] are key to establishing first order necessary conditions of optimality 
for Problem V. 
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APPENDIX B: A Differential Game Model for the Study of Optimal Air-War 

Strategies • 

1 . Introduction » 

An important question for defense planners is what are appropriate 
missions over the course of a tactical campaign for tactical air power. 

The answer to this question has far reaching implications for Navy air 
forces (both carrier-based and land-based) (and, of course, the Air Force). 
The quest for an answer to this type of question has been in large part 
the genesis of modern quantitative methods of military analysis. One has 
only to recall the classical work of Lanchester [10] or that reported in 
Morse and Kimball [11] (see especially pp. 73-77). More recently, the 
USAF Studies and Analysis Group has been using quantitative methodology 
[24] in trying to answer such questions. 

There are many analysis approaches to answering the question of 
the "best'’role (over time) of tactical air support. These range from 
operational gaming (see [12] or [23] for discussion of terminology and 
background) to analytical solution of an idealized differential game. 
However, as pointed out by L. Berkovitz and M. Dresher (see p. 612 of [1]), 
"operational gaming is not a helpful device for solving a game or getting 
significant information about the solution." We believe that there is 
much to be gained by a more analytical study of an idealization of the 
basic problem (see also [18] or [19]). Therefore, we will consider a 
simplified differential game model for the "best" (in the sense discussed 
in Section below) use of air power, with our eye on Navy applications. 

As we have noted in Appendix A and elsewhere [18], [19], and impor- 
tant gap in the previous theory of differential games has been the lack 
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of multiplier conditions for differential games with state variable 
inequality constraints (SVIC*s). It should be noted (see also [19], [21]) 
that state variable inequality constraints are present in all Lanchester- 
type dynamic tactical allocation problems because force levels (which are 
represented by state variables ) are required to be non-negative (or some 
equivalent condition). Thus, the theoretical state-of-the-art previously 
was not adequate to solve all such problems. Hence, this (our study of 
the below problem) is an important application of our new theoretical 
results given in Appendix A. It should be pointed out that the current 
state-of-the-art in optimal control theory (at least its application in 
operations research) has not even been sufficient to allow routine solution 
of Lanchester-type optimal control problems (see [19]) because of the pres- 
ence of SVIC^s (see [17], [22]). 

In this appendix then we will consider an idealization of a tactical 
air-war game. Versions of this problem have been considered by a number 
of workers [1], [2], [5], [6], and [7]. A particularly thorough discussion 
of modelling aspects of the problem is to be found in [6]. This is the 
best reference on modelling aspects that the author has encountered so far 
and is the source of many ideas. A special version of the problem that 
we will consider has been studied by R. Isaacs (see pp. 96-104 in [7]). 
Although he obtains a correct and thorough solution to the problem he 
considered, Isaacs does not employ the equivalent of optimality conditions 
for differential games with SVIC’s. The optimality of constrained subarcs 
is not examined, and Isaacs’ approach may not yield optimal strategies 
for more general problems. 
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A. Friedman (see pp. 239-240 of [4]) also considers Isaacs’ "War 
of Attrition and Attack." His treatment is not nearly as thorough as that 
of Isaacs, and Friedman’s solution is not complete. He (as does Sternberg 
[16]) misses certain subtle aspects first observed by Isaacs (see pp. 102- 
104 of [7] and also pp. 948-949 of [9]). Moreover, the emphasis of 
Friedman’s research [4] is on the existence of value for differential 
games with SVIC’s. The analogues of the well-known control theory multi- 
plier conditions for constrained subarcs [8] are not developed. In fact, 
the treatment of the example on pp. 239-240 of [4] by Friedman is inade- 
quate, since his approach fails to yield optimal strategies for minor 
modifications in the problem’s formulation (see below). Thus, in this 
appendix we present a different and more general analysis than has appeared 
in the literature previously. 

The problem that we study in this appendix is quite significant 
from the standpoint of military operations analysis. Currently, the USAF 
is using a similar (although much more detailed) model (TAG CONTENDER) [24], 
which has been widely used at the DOD/JCS planning level. Our research 
should shed light on cause-effect relationships between modelling assump- 
tions and optimal air-war strategies obtained from TAG CONTENDER. 

One may consider that there are four essential parts of any dynamical 
combat optimization problem: 

(1) the decision criteria (for both combatants) , 

(2) the model of conflict termination conditions (and/or unit break- 
points) , 

(3) the model of combat dynamics. 



(4) the decision latitude of the combatants. 
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By decision latitude we mean the extent of choices that decision makers 
on both sides can actually control. The other aspects have been discussed 
in [19]. Let us mention here that recently Pugh and Mayberry [15] have 
discussed the first topic (measures of effectiveness) (see also [14]). 

They also propose a computational scheme [14] (Lagrange dynamic programming 
[13]). Our analytical work here (and in the future) provides test cases 
for the adequacy of their computational approach. 

Our analysis approach is more idealized than the TAG CONTENDER model 
or the conceptual approach of Pugh and Mayberry [14]. However, we feel 
that our research complements these other efforts. We view that our 
differential game analysis has several significant uses: 

(1) provide guidance for higher -resolution studies, 

(2) identify cause-effect relationships between optimal tactics and 
modelling assumptions. 

2. The Model : A Generalization of the Tactical Air-War Game . 

We consider an idealized model in order to gain insights into optimal 
air-war tactics. Background material on this model can be found in [6] 

(or [1]). The problem that we consider is a generalization of the tactical 
air-war game studied by R. Isaacs [7] (i.e. his '’War of Attrition and 
Attack”). The origin of this problem is apparently due to A. Mengel (see 
[ 6 ]). 

Consider a war between X and Y forces. A land war takes place 
and its location will be denoted as the FEBA (P^orward Edge of the Rattle 
Area) . Both the X and Y forces have a single type of aircraft that 
can fly two types of missions: (1) ground support which means flying fire 



support missions for the ground forces to influence the outcome of the land 
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war and (2) counter-air which results in the shooting down of enemy planes 
(but no direct help for the ground forces) . The problem for each commander 
is to find the ’*best*' use of his air power (x(t) denotes the number of X 
aircraft) . The decision making criterion for each commander is the net 
amount of support for the FEBA measured in terms of the 'Value” of total 
missions flown (or, equivalently, weighted tons of ordnance dropped or 
weighted total ground support-aircraft days). For planning purposes, the 
air war lasts for a prescribed duration of time, denoted as T, and residual 
values of surviving aircraft (based on linear utility for both sides* sur- 
viving aircraft) is considered. It is assumed that new aircraft are 
introduced on both sides at constant rates. This situation is shown dia- 
grammatically in Figure 1. 

1-u 




Figure 1. Diagram of Generalization of Tactical Air-VJar Game. 



I am indebted to MAJ Ron Kronz , USAF for pointing out that this is commonly 
done in similar USAF analyses (i.e. TAG CONTENDER). 
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Mathematically the problem may be stated as 



maximize minimize {v x(t^)-v y(t ) + [B (t) ux-A(t) vy] dt} , 
U V X t Y t ^ 



with stopping rule: 



- T = 0, 



subject to: 
(air-battle dynamics) 



^ = r - (l-v)a(t)y, 



( 1 ) 



^ = 
d t 



s - (l-u)b(t)x. 



with initial conditions x(t=0) = x^ and y(t=0) = y^, 

and 

x,y ^ 0 (State Variable Inequality Constraints), 



0 ^ u,v 1 (Strategic Variable Inequality Constraints), 

where 

x(t) and y(t) are the numbers of X and Y aircraft, respectively, 
at t ime t , 

a(t) and b(t) are variable (Lanchester) attrition-rate coefficients, 
r and s are replacement rates, 

V (v ) is the value of one surviving X(Y) aircraft, 

X Y 

A(t) (B(t)) is the value of one Y (X) aircraft flying ground 
support for a unit of time at time t, 

u(v) is the fraction of total X(Y) aircraft which fly ground 
support missions* 



A discussion of the circumstances under which the above air-war equations 
have been hypothesized to apply can be found in [20]. It should be noted 
that capital letters are used to denote (closed-loop) strategies, i.e. 

U = U(t,x,y) and V = V(t,x,y), whereas lower case letters are used to 
denote the strategic variables which are their outcomes (or realizations 
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i* 

of the strategies), i.e. u = u(t) = U(t,x,y). Optimal (and also extremal ) 
strategies will be denoted as follows: U and V . 



3 . Development of Basic Necessary Conditions of Optimality . 

It should be clear that we have in (1) above that v ,v ^ 0, that 

A X 

r,s > 0, and that a(t) ,b(t) ,A(t) ,B(t) > 0 for t 6 [0,T]. Friedman has 
shown that the differential game (1) has value (see pp. 231-232 of [4]), 
and it may be shown that a saddle point exists in pure strategies (see pp. 
234-235 of [4]). We now will develop necessary conditions of optimality 
for (1). 

Applying the results of Appendix A, we have that the Hamiltonian is 
given by 



H(t,x,y,p,q,y ,n,u,v) = B(t)xu - A(t)yv + 

p{r-(l-v)a(t)y} + q{s-(l-u)b(t)x} - p(t)x + n(t)y, (2) 

where 

, = 0 for X > 0, 

p(t) I 

^ ^ 0 for X = 0, 

and 

! = 0 for y > 0, 

^ 0 for y = 0. 



We have adopted above the following correspondence between state and adjoint 
variables : 



state variable dual variable 

X P, 

y q. 



^We use the term extremal to denote a trajectory on which the necessary condi- 
tions of optimality (see Appendix A) are satisfied at every point in time. 
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It is also convenient to define 



$(t^,x(t^) ,y(t^) ,a) = v^x(t^) - v^yCt^) - a(t^-T). 



(3) 



The adjoint system of differential equations for the dual variables is 






dt 8x 



la = - M = 

dt dy 



B(t)u* + (l-u*)b(t)q + y, 



A(t)v^^ + (l-V'Oa(t)p - n* 



(4) 

(5) 



The boundary conditions at t = t^ for the adjoint variables are 



p(t=t^) = ~ '^1 



q(t=t^) = -v^ + where 



= 0 for x(t^) > 0, 



^ 0 for x(t^) = 0, 



= 0 for y(t^) > 0, 



^ 0 for y(t^) = 0. 



(6) 



(7) 



We also have the transversality condition 

“<h> 

which yields that a = H(t^) • This latter condition, however, is not useful 
for determining optimal strategies. 

When x,y > 0, the extremal strategic-variable pair, denoted as 
(u*,v*), is determined by the max-min principle . Hence, we consider 



maximize minimize H(t ,x,y ,p,q,y=0,n=0,u,v) , 
0^u:s:l O^v:^ 



( 8 ) 



which by (2) yields 



maximize xu{B(t) + b(t)q), 
0^u:s:l 



( 9 ) 
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and 



minimize y(-v){A(t) - a(t)p}. 
Thus for x,y > 0, we have 

u*(t) = < 

0 for (-q(t)) > 



and 



V*(t) 



1 for 



P(t) 



< A(t) 
a(t)» 



0 



for 



p(t) > 



A(t) 

a(t)* 



( 10 ) 



( 11 ) 



( 12 ) 



We must further investigate the possibility of singular subarcs (see 

9H 

Chapter 8 of [3]) on which, for example, = 0 for a finite interval 

o u 

of time (so that all its time derivatives vanish) . For a u-singular 

9H 

subarc, the condition that = 0 with x,y > 0 yields that on a u- 
singular subarc we must have 



B(t) 



b(t)* 



(13) 



The condition 



— f— ] = 0 

dt’-3u'' 



along with (12) then yields 



A(t)v* + (l-v*)a(t)p(t) = - ■^lb(t)^ ’ 



(14) 



or, equivalently, by (12) 

maximum {A(t) ,a(t)p(t) } = - 

Considering the adjoint equations, it is unlikely that (15) would hold for 
a finite interval of time. Moreover, when a(t) , b(t) , A(t) , and B(t) 
are all constants (e.g. a(t) = a = constant), it is immediately seen that 
(15) cannot hold, since 
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maximum {A,ap(t)} ^ A > 0 = - 



dt 






Thus, it is impossible (in general) to have a u-singular subarc of an 
optimal trajectory for (1). In a similar fashion, it is readily shown 
that it is impossible (in general) to have a v-singular subarc. 



3.1. Necessary Conditions of Optimality on Constrained Subarc for x . 

X X 

On a constrained subarc on which x(t) = 0 for t t t. the 

e a 

dx X X 

Y strategic variable v*(t) is determined by — = 0 for t < t < t„ 
° dt el 

and hence 



(1-v*) = 



a(t)y(t) ’ 



or 



v*(t) = 1 /Vh^Vt- ' V t^ < t < t^. 

a(t)y(t) e I 



(16) 



dx X— X“ 

Since we must have 0 v* 1 and — (t ) < 0 (here t denotes a 

dt e e 

left-hand limit) , such a constrained subarc can occur only when 



r ^ a(t)y(t) for t^ < t ^ t^, 



(17) 



XX XX 

with r < a(t )y(t ). Since x(t) = 0 for t t t„ , the dimension 
e e e Jc 

of the state space may be thought of as being reduced by one, and hence 



the dimension of the adjoint variable space is reduced by one through 



3H 

— = 0. This yields 



, s A(t) ^ X X 

=700 "e ^ ^ h- 



(18) 



d 9 H 

The multiplier u(t) is determined by ® hence 



M(t) = B(t)u* + (l-u*)b(t)(-q(t)) for < t < t^. (19) 
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Thus, the necessary condition of optimality y(t) s 0 leads to that on 
a constrained subarc with x(t) =0 for a finite interval of time we must 
have 

B(t)u* + (l-u*)b(t)(-q(t)) ^ for t^ < t < t^ (20) 

dt ‘‘a C t V e £ 

X X 

When y > 0 and x(t) = 0 for t^ t t'^^ , we have that v*(t) is 

dx 

determined by the condition that =0. It should be noted that (18) 
implies that (10) is identically satisfied. In this case only u* is to 
be determined by the max-min principle (8). However, when x(t) = 0, then 
the max-min principle (or, equivalently, (9)) no longer yields that we must 
have u*(t) given by (11). However, we may still take (11) to hold by a 
continuity argument in which we consider x(t) = € > 0 and then let € 0. 

Thus, (11) may be used to write (20) as 

maximum {B (t) ,b (t) (-q(t) ) } ^ *^e 

Remark : The significance of (20) is that the optimality of Y pursuing a 

strategy which results in x(t) = 0 for a finite interval of time 
depends upon what X’s optimal strategy for using the aircraft 
would be if he had them. This type of behavior in which the 
optimality of one player steering the system along a state-constrained 
subarc depends upon the other player's optimal strategy is unique 
to differential games. Furthermore, it apparently has not been 
noted in the literature previously. 

It remains to discuss the corner conditions which must hold at entrances 

to and exits from constrained subarcs on which x(t) = 0 for a finite interval 

X X 

of time. At entrance to constrained subarc with x(t) = 0 for t ^ t ^ t. 

e X 



(see Figure 2 below) , we must have 
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Figure 2. Entry to and Exit from Constrained Subarc. 



. X-, . X+. .X. 

P(te ) = P(t^ ) = p(t^) , 

q(t^') = qCt^"^) = q(t^), (22) 

H(tf) = H(tf), 

where H(t^ ) denotes H(t^,x(t^)=0,y(t^) ,p(t^ )>q(t^ ) ,y=0,n=0,u*(t^ ), 

Q. G G G G G G 

X— 

v*(te ) = !)• It is rGadily sggii that 

H(t^ ) = p(t^){r - a(t^)y(t^)} + q(t^)s. 



and that 
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H(cf) ■ -A(t^)y(t^)U - 



X, 



-} + q(t^)s, 



A(t^) 

so that H(t^ ) = HCt^"*”) =» p(t^){r - a(t^)y(t^)} = ^ {r - a(t^)y(t^)}, 

e e e ee /Av e e 



whence 



a(te> 



A / X, 

, X, 

■ TT 

a(t^) 



(23) 



d X X”“ 

since we must have TrCt ) < 0. We thus see that the corner conditions 

dt e 

(22) are automatically satisfied when (16) through (20) hold, and thus they 
yield no new information. 

X X 

At exit from constrained subarc with x(t) = 0 for t ^ t ^ t^ , 
e Z 



we must have 



. X-. , X+. . X. 

p(tj^ ) = p(t^ ) = p(t^) , 

H(tf) = H(tf). 



(24) 



It is readily seen that 



H(t^~) = -A(t^)y(t^){l - 



r 1 . / Xs 

xTTx:’ 



a(tpy(tj) 



and that 



TT / X+. , X. > X. > Xv / Xv 

H(t^ ) = -A(tjj^)y(t^) + (tj^)r + q(t^)s, 



y_ Y-L 

so that ^(t^ ) = H(tj^ ) => 



A/ Xx 

, X, 

■ “7 ir 

a(tp 



(25) 



Again, we see that the corner conditions (24) are automatically satisfied 
when (16) through (20) hold, and thus they yield no new information. 
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3.2. Necessary Conditions of Optimality on Constrained Subarc for y . 

Similar analysis yields for a constrained subarc on which y(t) = 0 
Y Y 

for t t t, that we must have 
e i 

=l-b(^ ( 26 ) 

Y Y Y Y 

with s ^ b(t)y(t) for ^ ^ ^ Additionally, 

we must have 

(-q(t)) = for ^ t ^ (27) 

and 

n(t) = A(t)v* + (l-v*)a(t)p(t) + ^0 for < t < tj. (28) 

The latter condition may also be written as 

A(t)v* + (l-v*)a(t)p(t) ^ ^ h’ (29) 

or by use of (12) (which was developed from (10) for y > 0) 

maximum {A(t) ,a(t) p(t) } ^ ~ ^^b~(' t l^ (^^^ 



The corner conditions yield that at entry to constrained subarc on 
Y Y 

which y(t) = 0 for t t t. , we must have 

e A* 

/ Y-. , Y+, . Y, 

) = P(tg) ’ 

q(t^’) = q(t^+) = q(J), (31) 



Also , 
which 



(-q(t^» = 

the corner conditions yield that 
Y Y 

y(t) = 0 for t^ t t^ , we 



B(t^) 

b(th 

e 

at exit from constrained subarc on 
must have 
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/ - Y+^ . Y, 

I ) = 

q(t^ ) = 



3.3. Discussion of Optimality Conditions for Constrained Subarcs . 

In this section we will discuss some implications of the above 
necessary conditions of optimality that must hold on constrained subarcs. 

Let us first consider the case in which the following hold: 

a(t) = a = constant, 

b(t) = b = constant, 

A(t) = A = constant, 

B(t) = B = constant. 

In this case, the necessary condition of optimality (21) which must hold 

XX X 

on constrained subarcs on which x(t) = 0 for t^ ^ t ^ t^ with t^ < t^ 

is automatically satisfied, since we have 

.maximum {B,b(-q(t))} ^ B > 0 = - • 

a 

Similarly, (29) (or equivalently (30)) is automatically satisfied on con- 

Y Y . Y Y 

strained subarcs on which y(t) = 0 for t^ ^ t ^ t^ with ^ Thus, 

* 

we see that necessary conditions of optimality are always satisfied for 
the holding of enemy aircraft at a zero level for a finite interval of time 
in the case of constant coefficients (i.e. the functions a(t) , b(t), A(t) , 

At least the ones that we have considered above. There are also, of course, 
conditions related to the transversality conditions (6) and (7) . These are 
examined in Section 4 below. 
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and B(t) are all constant) in (1). This is precisely the case of (1) 
previously studied by others [4], [7], [16]. In the case of constant 
coefficients, there is no question about the optimality of a policy which 
results in the annihilation 'of all enemy aircraft during the appropriate 
phase of the air war (see Section 5 below). However, in the case of time- 
dependent returns from ground support and time-dependent Lanchester attri- 
tion-rate coefficients, the optimality of such a policy is not always 
guaranteed. Furthermore, the optimality of a strategy which results in 
the annihilation of enemy aircraft depends upon the enemy’s strategy for 
using his aircraft if he did have them (see, for example, (20) above). 

More precisely, for example, the multiplier condition ]j(t) ^ 0 (upon 
which Y bases his strategy) which must hold on a constrained subarc on 
which x(t) = 0 depends upon X’s optimal strategy (i.e. the enemy’s 
strategy) . This behavior is unique to differential games and has apparently 
never been noticed before. 

Let us now give two simple examples which show the complexity in 

the general case of determining whether X should try to shoot down all 

enemy planes (during the appropriate phase of the air war). Thus, we 

consider the necessary condition of optimality (29) which must hold on a 

Y Y Y Y 

constrained subarc on which y(t) = 0 for t ^ t ^ with t < t„ . 

^ e a el 

In the first case, let us assume that the following hold: 

a(t) = a = constant, 
b(t) = b = constant. 



Then, (30) takes the form 



— 1 dB 

maximum (A(t),ap(t)) ^ * 

b 
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Let us assume that we have A(t) ^ ap(t) . Then we have for the appropriate 
phase of the air war 

(a) if ^ ^ 0, X should try to shoot down all the Y planes, 

(b) if X may want to fly all ground support missions now 

and shoot down enemy planes later in this case in which X*s rate 
of return from ground support is decreasing. 

In the second case, let us assume that the following hold: 

A(t) = A = constant, 

B(t) = B = constant. 

Then, (30) takes the form that we must have 



B db 



maximum (A,a(t)p(t)) ^ ^ » 



in order for it to be optimal to have y(t) = 0 for a finite interval of 
time. Then we have for the appropriate phase of the air war 

(a) if ^0, X should try to shoot down all the Y planes, 

dt 

(b) if ^ ^ to forget about shooting down Y aircraft 

today because tomorrow he will be so much more effective doing this, 



3.4. Introduction of the Switching Functions S (t) and S (t) . 
^ ^ 

It is convenient to consider the switching functions S (t) and 



S^(t) defined by 



S^(t) = (-q(t))b(t) - B(t), 



(33) 



and 



S^(t) = p(t)a(t) - A(t) . 



(34) 
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Then, we may re-write the extremal control laws (11) and (12) for the 
strategic variables as 
for y > 0 : 



/ 1 for 
u^(t) = I 

^ 0 for S^(t) > 0, 

and 



for X > 0 : 



v*(t) 



1 for S (t) < 0, 

V 

0 for S^(t) > 0 . 



(35) 



(36) 



Since we will develop the solution to (1) by working backwards from 
the end at t^ = T, it is convenient to introduce the backwards time x 
defined by 

X = T - t. (37) 



It should be noted that =- 7 ^. We have then from (33) and 

dt dx dx dx 



(34) 



and 



S^(x) = (-q(x))b(x) - B(x), 



(38) 



S^(x) = p(x)a(x) - A(x), 



(39) 



where A(x) = A(T-x) , etc. and p(x) = p(T-x) and we again denote 
p(x) as p(x), etc. Then from (35) 
for y > 0 : 



u*(x) = 



1 for 3^(x) < 0 , 



0 for ^^('^) ^ 



(40) 
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and it follows from (38) and the adjoint equations that 



dS 
u 

dx 



b(x){A(T)v’^ + (l~v^)a(T)p(T) } + (-qCx)^ - ^ for y > 0, 

dx dx 

0 for y = 0 (for a finite interval of time) . (41) 



Similarly , 

for X > 0: 

/ 1 for S (x) < 0, 

I V ’ 

V*(x) = j 

V 0 for S^(x) > 0, (42) 

and it follows from (39) and the adjoint equations that 



dS 

V 

dx 



a(x){B(x)u* + (l~u*)b (x) (-q(x) ) } + p(x)^ - ^ for x > 0, 
0 for X = 0 (for a finite interval of time) . 



4 . Extremal End States . 

By an extremal end state we mean a terminal point in the state space 
for which necessary ( transversality) conditions of optimality are satisfied. 
Let us first consider in detail the situation for the X aircraft. There 
are two possibilities which we denote as follows: 

(a) eJ : x(T) > 0, 



and 



(b) 



: x(T) = 0. 
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For , there are two further subcases as follows: 

(1) x(T) = 0 with x(t) = 0 for t^ ^ t ^ T where t^ < T, 

(2) x(T) = 0 with x(t) > 0 for T - 6 t < T where 6 > 0. 

Let us first consider the end state : x(T) > Q > By (6) at 

t^ = T, we have p(t=t ) = v , since x(t-=T) > 0. Thus, 
r r A 1 



for x(t^) > 0 : p(t=t^) = v^. 



Also, by (36) (or equivalently (12)) 



v*(t=t^) 



1 for 



X 



A(T) 

a(T)’ 



0 



for 



X 



A(T) 
a(T) . 



(44) 



(45) 



Comment : 



If V = 0, then v*(t=t^) = 1. This is the only case considered 

A 1 

in previous analyses [4], [7], [16]. 



Let us now consider necessary conditions of optimality for the end 

state E^ ; x(T) = 0 . As above, there are two subcases to be considered. 

X X 

In case (1) : x(T) = 0 with x(t) = Q for t^ ^ t ^ T where t^ < T 
(i.e. at the end the system has been on a constrained subarc for a finite 
interval of time) , we have by (18) that 

P(T) - fgf, (46) 

since the system has been on a constrained subarc for a finite interval 
of time. Also, by the transversality condition (6), we have that 

p(T) = V - V where v 0. | 
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By (46) and (47) , we have 



A(T) 

a(T) 



^ 0 . 



so that we must have 



X 



A(T) 

a(T)’ 



(48) 



for it to be optimal to have x(T) = 0 when the system has been on a 
constrained subarc for a finite interval of time. In case (2) : x(T) = 0 
with x(t) > 0 for all t € [T-<5,T) where 6 > 0 , it is clear that we 
must have v*(t) = o’*" for t € (T-6^,T) c (T-6.T) where 0 < 6^ 6, 

since this is the only way that we can reach x(T) = 0 with x(t) > 0 
for all t £ [T-6,T). By (10) (see also (12)) in order to have v*(t) = 0 
for all t € (T-6 ,T) we must necessarily have p(t) ^ . By the 

continuity of p(t) (see Appendix A) it is necessary that we have 



p(T) 



A(T) 

a(T)' 



(49) 



We also have the transversality condition (47) so that p(T) = v - v ^ 

A i 

A(T)/a(T) which we may also write as v ^ A(T)/a(T) + v ^ A(T)/a(T), 

X 1 

since ^ 0. Thus, we see that (48) is again a necessary condition for 
it to be optimal to have x(T) = 0 but x(t) > 0 for all t (: [T-6 ,T) 
where 6 > 0 . 

Considering the above, we see that we have proved the following 

theorem. 



THEOREM 1: 



If an optimal strategy for Y is to result 
in x*(T) = 0, then it is necessary that 



V 



X 



^ A(T) 
a(T)' 



It should be recalled that we have shown it to be impossible to have a 
singular solution for x,y > 0. Hence, v*(t) must be equal to 0 or 1 
almost everywhere in time. 
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In other words, in order that the end state : x(T) = 0 be an extremal 

end state we must have v ^ A(T)/a(T). We also have the following as an 

X 

immediate corollary. 

A f T) 

COROLLARY 1.1: If < — ^ -y, then an optimal strategy 

for Y leads to x*(T) > 0. 



Remark: In the problem considered by Isaacs [7] and others [4], [16] 

previously, we have v = 0, A(t) = 1, and a(t) = a = constant 

X 

so that an optimal strategy for Y resulted in x*(T) > 0. The 
optimality of this condition apparently was never examined pre- 
viously. In fact, an implicit assumption in Friedman’s example 
on pp. 239-240 of [4] is that an optimal policy results in 
x*(T) > 0. From Theorem 1 we see that this need not be true in 
general . 



In a similar fashion, we can prove the following theorem. 



THEOREM 2: If an optimal strategy for X is to result in 

y*(T) = 0, then it is necessary that 

B(T) 

'"y ^ b(T)' 



We also have the following as an immediate corollary. 



COROLLARY 2.1: 



If 

for 



Y 

X 



B(T) 

b(T) 

leads 



then an optimal strategy 
to y*(T) > 0. 



Remark : It seems appropriate to point out the following interpretations: 



(a) a(T)v = rate at which one Y aircarft can destroy value 

A 

of X aircraft at end of campaign, 

(b) A(T) = rate of return from one Y aircraft from flying 
ground support missions at end. 

Considering the above, the meaning of Theorem 1 is clear: in 

order for Y to follow a strategy which results in x(T) = 0, 
it must be to his advantage to do so. 



5 . Solution for Special Case of Constant Coefficients . 

In this section we will develop the solution to (1) for the special 
case in which all the coefficients are constants. In other words, we will 
consider the case in which the following hold: 

a(t) = a = constant, 

b(t) = b = constant, 

A(t) = A = constant, 

B(t) = B = constant. 



For notational convenience we will again denote a as a, etc. 
then convenient to re-state the problem as follows: 



maximize minimize {v x(t-)-v y(t^) + 
U V X t Y t 



(Bux-Avy)dt} , 



It is 



with stopping rule: t^ - T = 0, 



subject to: 
(air-battle dynamics) 



— = r - (l-v)ay, 



(50) 
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with initial conditions x(t=0) = and y(t=0) = 

and 

x,y ^ 0 (State Variable Inequality Constraints), 

0 u,v 1 (Strategic Variable Inequality Constraints), 



where a is a given constant and similarly for b. A, and B. 

Let us now review the necessary conditions of optimality that we 
have developed for (50) in Section 3. Let us recall the switching functions 
in backwards time x = T - t. These are 



S^(x) = b(-q(x)) - B, (51) 

and 

S^(x) = ap(x) - A. (52) 

We had developed earlier (see (40) through (43)) the following character- 
ization of optimal strategies: 
for y > 0: 



u*(x) = 



1 for S^(x) < 0, 



0 for S (x) > 0, 
u 



(53) 



and 



for X > 0: 



v*(x) = 



1 for S (x) < 0, 

V 



0 for S^(t) > 0, 



(54) 



with 



dS 
u 

dx 



b{A^* + (l-v*)ap(x)} > 0 for y > 0, 

( 55 ) 

0 for y = 0 (for a finite interval of time) , 



B-25 



and 

a{Bu* + (l-u*)b(-q(x)) } > 0 for x > 0, 

( 56 ) 

0 for X = 0 (for a finite interval of time). 

dS 

Thus we see that S 0 with strict inequality for y > 0 and similarly 

for S^(t). From (15) we see that singular subarcs are impossible. 

X X 

On a constrained subarc on which x(t) = 0 for t < t < we 

e X 

have shown the following to be necessary conditions for optimality: 



v*(t) =1 

ay(t) 


for 


X 




X 


(57) 


t < 
e 


t < 




r < ay(t) 


for 


t^. 

e 


t < 


X 


(58) 


P(t) =- 


for 


t^. 

e 


t < 


X 


(59) 



The requirement (21) is automatically satisfied, since 

maximum {B,b(-q(t))} > B > 0 = - for t^ < t < t^. 

dt^a^ e % 

Similar consitions hold on a constrained subarc on which y(t) = 0 for a 
finite interval of time. 

Concerning extremal end states, we have developed the following. 




dS 

V 

dx 
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PROPOSITION 3: If an optimal strategy for X is to result 

in y*(T) = 0, then it is necessary that 




PROPOSITION 4: If then an optimal strategy X 

1 b 

leads to y*(T) > 0. 



We also have the following boundary conditions for the adjoint variables: 



p(t=T) ~ ~ '^l 



= 0 for x(T) > 0, 
> 0 for x(T) = 0, 



(60) 



q(t=T) = + ^2 ^^2 



= 0 for y(T) > 0, 
> 0 for y(T) = 0. 



(61) 



We may then write the differential equations for the switching functions 



as 



and 



/ b{A + (l-v*)S (t)} for y > 0, 

5T- 



(62) 



0 for y = 0 (for a finite interval of time). 



/ a{B + (l-u*)S (x)} for x > 0, 

dS I u 

dx I 



(63) 



0 for X = 0 (for a finite interval of time). 



with initial conditions 



S (t= 0) = (bv^-B) - bv^ and S (t= 0) = (av -A) - av . 

U 1 ^ V A .1 



( 64 ) 
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5.1, 



Synthes is of Extremal 



There are four cases 

/T\ A 

"X ' a 



S trategic- Variable Pair . 
to be considered: 



(II) 



X 



> 



A 

a 



( 1 ) ( 1 ) 

( 2 ) ( 2 ) 

These cases will be denoted as (II), (12), (III), and (II2). The solution 
to (50) must be developed for each case separately. It should be noted 
that Case (12) and Case (III) are symmetric: Case (12) is the same as 

Case (III) only with the roles of X and Y interchanged. These then 
are three cases of different structures for the optimal strategies. 

In each case, the solution to (50) is developed by working backwards 
from the end of the war at t = t^ = T (or, equivalently, T = 0) where 
the boundary information (60) through (63) is known. The differential 
equations for the switching functions ^^(*^) then be 

integrated backwards (using (53) and (54) (or (57) on constrained subarcs)) 
so that the initial conditions are just met. This procedure leads to syn- 
thesis of an extremal strategic-variable pair and a corresponding extremal 
* 

trajectory. In the case in which the extremal trajectory is unique (in 
the sense that there is a single extremal path through any point of the 
state space which leads to the terminal manifold, it is the optimal trajec- 
tory. This is the situation in all cases except for one exceptional set 
of circumstances for Case (12). Tlie optimal strategies in this (exceptional) 
situation liave not been resolved at the time of the writing of this report. 
This is an important future research task, since the development of 



By an extremal trajectory we mean a path along which the necessary conditions 
of optimality (see Appendix A) are satisfied everywhere in time. 
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computational schemes for time-sequential combat games (such as application 
of Lagrange dynamic programming as proposed in [13], [14]) depends upon 
its successful completion. 



^ /-r ^ A , B 

Case (II): < — and v„ < - , 

V L 3 Y b 



Let us make the nonres trictive assumption that 



A-av, 



X 



B-bv^ 



aB bA 

By Propositions 2 and 4 we have that x*(T) > 0 and 
by (64) we have 



y*(T) > 0 



(65) 
so that 



S (t= 0) = bv - B < 0 and S (t= 0) = av - A < 0. (66) 

U 1 V X 

One obtains from (66) via (53) and (54) that 



u*(t=0) = 1 and v*(x=0) = 1. 



(67) 



By straightforward continuity arguments, it is readily seen that 



u*(x) = 1 

for X e [0,x ] , . 

v*(x) =1 



( 68 ) 



where x^^ denotes the "backwards time" of the first switch in (extremal) 
tactics. Using (62), (63), and (68), we see that the differential equations 
for the switching functions are for 0 S x < x^^ 

dS 

= bA with S (x=0) = bv„ - B < 0, (69) 

dT u Y 



and 



dS 

= aB with S (x=0) = av_, - A < 0, (70) 

dx V X 
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so that for 0 < x < t 



S^(t) = bAx + bv^ - B, 



(71) 



and 



S (x) = aBx + av - A. 

V A 



(72) 



The "backwards" switching time t is determined by S (t=t ) = 0 and 

u u u 



hence 



B-bv, 



T = 



u bA • 



(73) 



Similarly, we have 



A-av 



T = 
V 



X 



aB 



(74) 



The assumption (65) then yields that x, = t < t so that v*(x) switches 

1 V u 

first in backwards time at 



A~av 



T, = T = 



X 



1 V aB * 
We then have for x(x) > 0 



(75) 



u*(x) = 1 
v*(x) = 0 



for X e 



(76) 



where X 2 denotes the "backwards time" of the second switch in (extremal) 
tactics. It should be noted that by (50) and (76) we have y(t) > 0 for 
0 < X < X 2 * Using (62), (63), (71), (72), (75), and (76), we see that the 
differential equations for the switching functions are for x^ < x < x^ 



dS 

- 7 -^ = b{A+S (x)} with S (x=x,) = bA{ 

d T V u J 







-B-bv^’ 


aB 




bA ^ 



} < 0 , 



(77) 



and 
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dS 

\ 

dx 



aB for X > 0 



for X = 0 (for finite 
interval of time) 



with 






= 0 . 



(78) 



There are two cases to be considered: 



(a) x(t=T“T^) > 0, 

(b) x(t=T-x^) = 0. 

In Case (a): x(t=T-Xj) > 0 , we have that x(x) > 0 for 0 ^ t T 

by (50) and (76). Hence, (54) and (78) yield that v*(x) = 0 for all 
X > x^. Integrating (77) and (78), we obtain 






2B'-a^ 





'B-bv^" 


aB J 


bA J 






(79) 



and 



S (t) = aB(T-T,) = aBx - (A-av ) . 

V J. A 



(80) 



The switching time x = x„ is determined by S (x=x ) = 0 and hence 

u 2 u u 



j 7B-bv A ,'A-av •> 

^2 ■ "u ■ SivA" + 2aAB|[-— )-(— J-]} - avjj} > . 



(81) 



Again, (53) and (55)' yield that once u*(x) = 0 there are no further 

switches in the strategic variables as we progress backwards in time. 

In Case (b) : x(t=T-X j^ ) = 0 , we have that x(x) = 0 for x^ = x^ 

X X 

< X < X and x(x) > 0 for x < x < x^, where denotes the 

e e z z 

’^backwards time” of the second switch in (extremal) tactics in the case 

X X 

in which x(x) = 0 for x. < x < x 5^ x^. When x(x) = 0 for x„ = x. 

1 e 2 X. 1 

X 

< X < x^ < we obtain from (77) and (78) that 
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S (t) = bA{x - 



B“bv, 



y1 



}> 



and 



S (t) = 0 . 

V 



X X 

When x(t) > 0 for x < x < x«, (77) and (78) become for x 

^2. G 



dS 

■^ = b{A + S^(t)} with S^(t=t^) = bAjx^ - 



/B“bv, 



Y) 



bA 



and 



dS 

= aB with S (t=t^) = 0, 
dx V e 



X % 

Integration of (84) and (85) yields that for t < t ^ 

e 2 



S (t) = (t-t^)^ + bA(T-T^) + bAjr^ - 

u Z e e '• e 



rB-bv, 



bA 






and 



S (x) = aB(x“T ) 

V ^ e 



Tlie switching time x = To is determined by S (x=t ) = 0 and 

u 2 u u 






for 



A-av„, 


X 'v, 


(-B-bv^-v 


[ XI 


Y 


1 aB 

1 J 


- T-, < T < - 

1 e 2 


bA 

k. / 



Thus, we have x^ = 



'v. X 

readily seen that x^ is a strictly increasing function of x 

/B-bv, 2 e 



X 



bA 






since we have 



9t. 



9t 



/ 



<^B-bv, 



a 2 - 2aAB{r^ - 



bA 



)' 



(82) 

(83) 
s T S T 

2 

(84) 

(85) 

( 86 ) 

(87) 
hence 

( 88 ) 

). It is 
for 



1 



(89) 
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8t 



It is readily seen that 



B-bv, 



bA 



3t 



9t, 



9 Y 

I > 0 for T < 
X 0 



2 X 

, and that — ;r < 0 for t > 
X e 



B-bv^ 

bT 

rB-bv, 



3t. 



, that 



3x 



X 



= 0 for 



3t 



Y\ 

:t 7“ • Hence, x has a 
bA I 2 



maximum (which we denote as xp for x^ 
to note that 






It seems appropriate 



T = T 
2 2 



{/a 2 + 2aAB{ 



'B-bv^l 


'A-av ^ 


bA J‘ 


. aB J 



} - av } for . (90) 

X el 



and 



'V. 



_ _ 
^2 ^2 



B-bv, 



bA 



for 



X 

T = 

e 



rB-bv^) 



bA 



Thus, we see that 



< x^ < x' = 



B-bv. 



— 

bA ' ’ 



(92) 



[ aB J ■ 



for 



< < 
e 



B-bv^-i 



bA 



. This means that the switching time for 



u*(x) depends upon force levels, since it depends upon whether or not x(t) 

rA-av, 



is driven to zero before t=T-Xj=T- 



X 



aB 



when T > x^* Further- 



XX X 

more, when x(t) = 0 for T-x =t ^t^tfl=T-x. , the switching 

e e ^ 1 

time for X, i.e. ^2 = x^, depends upon the time (or, equivalently, the 

backwards time x ) at which the X forces are annihilated. Again, (53) 
e 

and (55) yield that once u*(x) = 0 there are no further switches in the 
strategic variables as we progress further backwards in time. This situation 
is shown diagrammatically in Figure 3 below. Thus, we see that X switches 
to concentration on ground support earlier in those compaigns in which 



Xq, y^, and T are such that X will lose all his planes before 
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Note: Strategic Variable s 

X: u 

Y: V 

Figure 3. Dependence of X Switching Time T 2 “ 
Upon Force Levels (not d rawn t o scale ) . 
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T - 



/-A-av >1 
X 




rb-bv 1 


> 0 but after T - 


I 


aB 


bA j 



> 0, T?iis means that X’s optimal 



strategy is to use his planes for ground support earlier In such cases than 
when x(t=T-T^) > 0 because they are annihilated before the last phase of 
the air war (when both sides concentrate on ground support). Thus, X uses 
his planes for ground support while they are still available. This phenom- 
enon was first noted (for a simpler case of (50)) by Professor Rufus Isaacs 



[7j (see also pp . 948-949 of [9]). 

A B 

Case (12) : v < -- and > —. 

_ ^ g Y 

By Proposition 2 we have that x*(T) > 0 so that by (64) we have 



S^(t= 0) = av^ - A < 0, 

and by (54) we obtain 



(93) 



v*(x=0) = 1. (94) 

By Proposition 3 an optimal strategy for X can lead to y*(T) =0 so 
that by (64) we have 



! = 0 for y(T) > 0, 

(95) 

> 0 for y(T) = 0. 

There are two cases to be considered: 

(a) y(t=T) > 0, 

(b) y(t=T) = 0. 

In Case (a) : y(t =T) > 0, we have that 



The reader should recall the nonrestrictive assumption (65) . 

This behavior is not noted in either [4] (see pp. 239-240) or fl6]. 
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S^(T=0) = bv^ - B > 0. 



(96) 



By continuity of y(t), we have that y(t) > 0 for t e (T-6,T] where 

6 > 0 so that (55) and (96) yield that S (t) > 0 for 0 < t < T. Hence, 

u 

(53) yields that u*(t) = 0 for 0 S t S T, and a backwards integration 
of the state equation for y(r) yields y(x) > 0 for 0 S t < T. Also, 
it is readily seen by a straightforward continuity argument on S^(t) that 



, u*(t) = 0 
' v*(t) = 1 



for T e [0,Tj^], 



(97) 



where denotes the "backwards time" of the first switch in (extremal) 

tactics. It is readily seen that x(t) > 0 for t e [0 ,t^). The dif- 
ferential equation for the switching functions are then for 0 S t < 
dS 



— ^ = bA with S (t= 0) = bv„ - B > 0, (98) 

dr u Y 

dS 

— ^ = a{B + S (t)} with S (T=0) = av^ - A < 0, (99) 

dr u V A 



so that for 0 < x < 

S (x) = bAx + bv - B, (100) 

U 1 

and 



S 

V 



(t) 



The switching time x^ 



a^ 

2A 



(Ax+Vy)2 - f v2 
is determined by 



+ av - A, 

A 

S (x=x ) = 0 

V V 



( 101 ) 

and hence 



X 



1 



X 

V 




(A-av^) - Vy} . 



( 102 ) 



There are two further subcases to be considered: 
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( 1 ) 

( 2 ) 

In Subcase (1): x(t=T-T j^ ) > 0 . 

S (t) > 0 for T > T, . Hence 

V 1 , 



x(t=T--r^) > 0, 
x(t=T-x^) = 0. 

continuity arguments on x(t) 



lead to 



/ u*(t) = 0 
^v*(t) = 0 



for T € (t^,T], 



(103) 



and x(t) > 0 for 0 t T. In Subcase (2): x(t=X-T j^ ) = 0 , we have 

XX X 

tliat x(t) = 0 for x. = < t x < T and x(x) > 0 for x < x < T. 

3c ± e e 

Using (62) and (63), we readily obtain for ~ ^ ^ 

S^(x) = bAx + bv^ ^ B > 0, (104) 

and 

S^(x) = 0. (105) 

Thus we have 



u*(x) = 0 

v*(x) =1 

ay 



for 



X c 



(t 



1 




(106) 



Since x(x) > 0 
that S (x) > 0 

V 



X X 

for X e (x ,x +6), arguments similar to the above yield 
e e 

and x(x) > 0 for x < x ^ T. Hence 
^ e 



u*(x) = 0 

for X £ (x^,T] . 

e 

v^(x) = 0 



(107) 



Case (b) : y(t=T) = 0 , there are two further subcases to be 



considered : 
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(1) 


y(t) = 0 for 


Y 

t < t < T 
e 


Y 

with t < T, 
e 






(2) 


y(t=T) = 0 but 


o 

A 

U 


for T - 6 < 


t < T where 6 


> 0. 


In Subcase (1): y(t) = 0 


Y 

for t < 


t < T with 


Y ^ . 

t < T, It IS 


readily 


seen that 


we must have 


e 




e 






/U*(x) = 




for T € 




(108) 




^v*(x) = 


1 









so that a result analogous to (59) , (62) , and (63) yield that for 
Y 

0 < T < T < 

e 1 



S^(t) = 0, (109) 

and 

S (t) = aBi + av ~ A. (110) 

V X 

Y 

It is readily seen that we must have y(r) > 0 for < t ^ T. Thus 



/ u*(t) = 0 
v*(t) = 1 



for 



T e 



r A 



( 111 ) 



where denotes the "backwards time" of the first switch in (extremal) 

Y 

tactics in the case in which y(x) = 0 for 0 ^ x x^, and 



u*(x) = 0 
* v*(x) = 0 



for X e (Tj^.T] , 



-x/ Y 

by (54) and (56), since y(x) > 0 for ^ 

Y 'V 

become for x < x < x, 
e 1 



( 112 ) 



Then (62) and (63) 
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dS 



u 



dT 

dS 



= bA 



with S (t=t ) = 0, 
u e 



dT 



= a{B+S (t)} with S (t=t ) = aBt + av„ - A < 0, 



(113) 

(114) 



Y 

so that for t < t < t, 
e 1 



S (t) = bA(T-T ) , 
u e 



(115) 



and 



abA . Y, 2 . Y, . „ Y 



S (t) = — r— (t-t )^ + aB(T-T ) + aBr + av^ - A. 
V 2 e e e X 



(116) 



The switching time t = x, is determined by S (t=t ) = 0 and hence 

V 1 V V 



Y ^ 1 r /, 

X = X = X + J 1 
1 V e bA'^V 



B^ - 2bAB{x^ - 



A-av 



aB 



} - b}, (117) 



Y 

for 0 < T < T- = 
e 1 



rA-av 



aB 



'Xj 'Xj Y 

. Thus, we have is readily seer 






rA-av 



that T is a strictly increasing function of x for 0 < x < 



aB 



since we have 






= 1 - 



9x 



/ V rA-av -N 

/ - 2bAB{T^ — 1 



(118) 



Of 

9x. 



'1 Y 

It is readily seen that — ~ > 0 for x^ < 



8x 



A-av 



X 



aB 



9x. 



, that 



= 0 for 



8x 



Y 

X = 

e 



A-av 



X 



aB 



9x^ 



'1 Y 

, and that — — < 0 for x > 
Y e 



3x 



A-av 



X 



aB 



Hence, x^ has a 



'^1 Y 

maximum (which we denote as x^ for x^ = 



A-av 



X 



aB 



, It seems appropriate 



to note that 



'U) 1 i / 

■ EaV 



+ 2bAB 



rA-av, 



x"^ 
e 



aB 



- b}. 



(119) 



and 



'Xj 'V, 

"i = "i = 





for 


Y 


'A-aVx^ 


aB J 


X — 

e 


aB J 



( 120 ) 
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It should be noted that 






rA-av. 



X 



aB 



( 121 ) 



^ A 

—\. As we discussed for Case (II): v,, < — and 

a B j X a 



for 0 < T < 
e 

g 

v„ < — , this means that the switching time for v*(t) depends upon force 
1 b 

levels, since it depends upon whether or not y(t) is driven to zero before 
t = T. Other aspects so far are similar to this previous case. 

In Subcase (2); y(t=T) = 0 but v(t) > 0 for T - 6 < t < T 
where 6 > 0 . it is clear that we must have u*(x) = 0 for all 
T e (0,6^) c (0,6). Thus, we have 



u*(t) = 0 
v*(t) = 1 



for T e [0,T^], 



( 122 ) 



where denotes the "backwards time" of the first switch in (extremal) 

tactics in the case in which y(i=0) = 0 but y(x) > 0 for 0 < t < 6. 
Considering (122), (64) yields that 



S^(t= 0) = (bv^-B) - bv 2 ^ 0, 



where 



0 < < 



bv^-B-* 



(123) 



(124) 



Using (122) in the integration of the differential equations (62) and (63) 
for the switching functions with the boundary conditions (93) and (123), 
we obtain for 0 < t < t. 



and 



S^(t) = bAj + bv^ “ B, 



" ||{At+(Vy-V2)}2 - 



ab 



(125) 



( 126 ) 



It should be noted that x(x), y(x) > 0 for x > 0. 
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The switching time x = T-, is determined by S (t=t ) = 0 and hence 
® V 1 ^ V V 






for 0 < V 2 5 



bv^-B 



. It is readily seen that 



, u*(t) = 0 
^ v*(t) = 0 



for T c (t^ ,T] . 



(128) 



Thus, we have where is chosen so that y(t=T) = 0 but 

y(t) > 0 for T - 6 < t < T. It is readily seen that x is a strictly 

rbv -B'. 



increasing function of for 0 < ^ 



, since we have 



8t 



i-iu- 



(Vy-Vj) 



avj A 






J > 0. 



(129) 



It seems appropriate to note that 



^1 * ’i ■ 



2A/, ^ 

^<A-aVjj) - 



for V 2 = 0, 



and 



— ^ O If/ 

^1 = ■'l =bXV^+2bAB 



cA-av„i 




rbv -B'l 


r1 


for V 2 = 


Y 


aB J 


b 



(130) 



(131) 



Thus, x^ is a strictly increasing function of V 2 , and we have 



- 'Vo 

X < X < X 

1 “ 1 “ 1 



for 0 < < 



bv^-B 



(132) 



It should also be noted that x^^ > x^ for bvY > B, Thus, from (132) we 
see that for bv^ > B more than one extremal can lead to a point = 

(Xf,Yf) = (x^,0) with x^ > 0 on the terminal manifold. Thus, in Figure 
4 the extremal trajectories labelled as (1), (2), and (3) all lead to the 
terminal manifold with = 0. Let = (^q^Yq) denote a point in the 
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initial state space from which an extremal leads to = (x^,0) with 



,u*(t) = 0 

0 S t < T and < T < Furthermore, let denote 



'Xy O 
^1 • 



the set of all such points, i.e. C° = {P°lsystem reaches P^ = (x^,0) from 
(U*(t) = 0 ^ ^ 'X/ 0*1 

iv*(t) = 1 0 < t < T and ^ ^ ~ ^ the backwards synthesis 

procedure for the extremal strategic-variable pair, it is readily seen that 
the set C lies on the "backwards extension" of the extremal which originates 

(in backwards time) from (x^,0) with x^ > 0 . The curve is shown in 

* o 1 u* ( t ) =0 

Figure 4 . From any point on C an extremal with ] , for 0 ^ t ^ T 

(t) = 1 

leads to P = (x^,0) by the appropriate choice of v^. For different values 



of P , there are different values of v. 



'Xy o 



One such P for which v is 
dS ^ 

V 



such that *^1 ^ \ shown in Figure 4. Since > 0, we must 

have v*(t) = 0 for t > so that working backwards yields that the 

extremal labelled (2) passes through P°. Thus, the figure suggests (also 

look at extremals emanating from P°) that there are several ways to reach 

the terminal manifold from a subset of the state space. The optimality of 

extremal trajectories has not been determined for this special case of 

multiple extremals. We will report our results to date with the final deter- 

mination for the complete solution to (50) in Case (12) : v < — and 

A a 

g 

v,^ > ~ being deferred to the future. In other words, the solution has not 
lb 

been completely determined in the shaded region of Figure 4. 

In considering Figure 4, let us consider the case in which we have 
/ u*(t) = 0 
^ v*(t) = v(t) 



for 0 < t < T. 



(133) 



The reader is cautioned that Figure 4 represents a two-dimensional (i.e. t 
and y) illustration of the pro jec tion of extremals onto a plane parallel to 
the plane x = 0 in the three dimensional state space (i.e. t, x, and y 
space). For notational convenience, we are not being precise about which 
entities are in the (t ,x,y)-space and which are projections onto the above 
noted subspace. Thus, denotes both a figure in (t , x,y)-space and also 

its projection on (t,y)-space. Similar remarks apply to Figure 5. 
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I 



Figure 4. Dependence of Y Switching Time = x^ 
Upon Force Levels for bv^ > B ( not drawn 




to scale) . 
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Then (133) may be combined with the state equations (50) and integrated to 
yield 



x(t) =XQ+rt-a 



and 



y(t) = y^ + St - b 



0 

t 

0 



{1 - v(s) }y (s)ds. 



X (s)ds . 



(13A) 



(135) 



The above may be combined to yield the following (Volterra integral equation) 
for y(t): 



y(t) = Yq + (s-bx^)t - ^t^ + ab 



0 ^ 



t s, 

ds 

J 1 J 
0 0 



{l-v(s2) }y(s2)dS2 (136) 



When v(s) = 0 for 0 < s < t, we obtain 



y^ °(t) = Yq + (s-bxQ)t - ^t2 + ab 



t s, 

ds. 



) J 
0 0 



y(s2)ds2, (137) 



(138) 



and when v(s) = 1 for 0 < s S t, we obtain 

Y'^~^(t) = yO + (s-bxQ)t - ^t^. 

It is also readily shown by direct integration of (50) with (133) when 

v(s) = 0 for 0 < s < t that 

- 

y^ (t) = — (ay^-r)cosh>^al^ t + ( )sinh/aF^ ^ T* 

^ ^ v/ab ^ 

Considering (137) and (138), it is clear that 

y^ ^(t) > y^ ^(t) for t > 0, (lAO) 



(139) 



provided that y(t) > 0 and that strict inequality holds for a finite 
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interval of time. Hence, whenever an (initial ) point lies on (such 

as (B) = in Figure 5), if Y uses the extremal strategy v*(t) = 0 

for 0 = T-T^<t<X-Tj^ when bv^ > B, the resulting extremal tra- 
jectory lies (strictly) above that corresponding to v*(t) = 1 for 
0 < t < T. This situation is shown in Figure 5 (which is expanded detail 
of Figure 4) below. Thus, at point (B) = (which lies on C^) in 

Figure 5 Y can use an extremal strategy which leads either in the direc- 
tion denoted as (3) or (4) (and results in extremals denoted as (b) and (c)). 
Moreover, if the trajectory began with t = 0 at, for example, point (A), 

then the extremal strategy v*(t) = 0 for 0 < t < T - must have been 

dS 

used, since if extremal (c) is chosen, S (t=t^) > 0 and > 0, and 

if extremal (b) is chosen, S (t=t^) = 0 and > 0. Hence, C itself 

V 1 dx 

may be a transition surface (depending upon the choice of Y when is 

first encountered). Furthermore, at (A) Y can choose either of two 
extremal directions denoted as (1) and (2) (and resulting in an extremal 
denoted as (a) or the one which leads to (B)). Thus, the problem apparently 
has multiple extremals (in the "triangular" region denoted as P°PQ) . It 
appears as though there may exist extremal trajectories (such as the one 
discussed above) at various points along which Y has a choice of alterna- 
tive extremal strategies (i.e. a bifurcation occurs) that result in extremal 
trajectories denoted as (a), (b) , and (c). 

It has not been determined (by direct computation of the criterion 
functional) which extremal strategy is optimal for Y in this special set 

The diagrams of battle trajectories in this report are sketched for a forward 
evolution of the battle (e.g. a curve of y(t) for constant ^/^) • Isaacs [7] 
considers a backward evolution (e.g. a curve of y(x) for constant x^ = x(t=T)) 
Neither approach is sufficient to determine optimal strategies. One must either 
invoke sufficient conditions for optimality or use the solution methodology 
referenced in [19] (see also [18], [21]). We hope to do this in the future. 
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Figure 5. Illustration of Multiple Extremal Strategies 

V* for Y when bvY > B ( not drawn to scale ) . 
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of circumstances. However, considering weLl-known sufficient conditions 
of global optimality based on convexity for optimal control problems, the 
author conjectures that all extremal strategies are optimal for Y and 
lead to the same return. If this is true, it means that there is an 
"indifference" set in Y’s optimal strategy space such that he can vary 
the survivability of his aircraft at will without changing the value of his 

g 

criterion functional. It should be noted that this occurs for v„ ^ — , 

1 b 

i.e. the Y aircraft (speaking somewhat imprecisely now) are valued greater 

than the return obtained from gound support missions at the end of the war. 

Case (III) : v,^ > — and v^, < — . 

, X ' a 1 b 

A B 

This case is similar to Case (12) (v„ < — and v^ > — ) with the 

X a lb 



roles of X 


and Y 


being interchanged. 


Case (II2) : 


V > — 

— X a- 


g 

and V,, > — . 
Y b 



By Proposition 1 an optimal strategy for Y can lead to x*(T) = 0 
so that by (64) we have 



S (t=0) = (av -A) - av-, where v 

U XI 



= 0 for x(T) > 0, 
> 0 for x(T) = 0. 



There are two cases to be considered: 

(A) x(t=T) > 0, 

(B) x(t=T) = 0. 

Similarly, by Proposition 3 an optimal strategy for X can lead to y*(T) = 0 
so that by (64) we again have (95). There are two further cases to be con- 



sidered : 



(a) y(t=T) > 0, 

(b) y(t=T) = 0. 
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The tedious treatment of the above four cases (i.e. (Aa) , (Ab) , (Ba) , 

and (Bb) where (Aa) denotes that we have x(t=T) > 0 and y(t=T) > 0 

is similar to that done in Case (12) (v^ < - and v^ > ^) and therefore 

A a lb 

the details will be omitted. It may be shown that in all cases (i.e. (Aa) , 
(Ab) , (Ba), and (Bb)) we must have 



S^(t= 0) > 0 and S^(t=0) > 0, 



(142) 



so that, for example, we have 0 ^ < 



raVjj-A. 



when x*(t=T) = 0. Con- 



sidering (53), (54), (62), (63), and (142), arguments similar to those 
used above readily yield 

( 0 for y(x) > 0 



u*(t) = 



1 - for y(x) = 0 



0 



for x(t) > 0 



v*(t) = 

Extremals are unique and therefore optimal. 



1 - — for x(t) = 0 
ay 



for all T e [0,T]. (143) 



5.2. Optimal Air-War Strategies . 

In this section we will summarize the results of the previous section 
In summarizing the optimal air-war strategies for X and Y, we must 
consider the four cases (i.e. (II), (12), (III), and (II2)) delineated above 
Case (II): av^ < A and bv^ ^ ^ < B . 

We make the nonrestrictive assumption that 
A-av, 



/A-av„^ 


/-B-bv^>k 


^1 < 


... _x 


[ aB j 


bA j 



(1) If T < 



X 



aB 



U*(t,x,y) = 1 
V*(t,x,y) = 1 



for 0 < t < T. 
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A-av. 



(2) If 



X 



aB 






A2+2aAB{ 



'B-bv^' 




.A-aVx^ 


bA J 




aB 



} - 



av 



x}’ 



U*(t,x,y) = 1 



1 

I V*(t,x,y) = j ^ 



ay 



and 



U*(t,x,y) = 1 
V*(t,x,y) = 1 



(3) If T > ^{yA2+2aAB{ 



for 


X > 


0 


for 


X = 


0 


1 








for 


T - 


1 






^B-bv^^ 




^A-av^' 


bA j 




aB J 



for 0 < t < T - 



A-av 



aB 



rA-av 



X 



aB 



< t < T. 



} - av } = T 



U*(t,x,y) = 



V*(t,x,y) = 



0 

1 

0 



s 

bx 



r 

ay 



for y > 0 
for y = 0 
for X > 0 
for X = 0 



for 0 < t < T - 



where 



(a) = ’’^2 x(t=T-Xj^) > 0, 



(b) T, = Tg + ^{/a +2aAB{ 



B-bv 



bA 



- T 



- A for x(t=T-x^) = 0, 



X X 

where x is largest value of x such that x(t=T-x ) = 0 

A-a§^ . , X , , '-'’''y 

—IT ■ 'l - "e - "2 ■ ~b^ ’ 

(c) T„ = xl for x(t=T-x,) = 0 when x^ > x', 

2 2 1 e 2 



also 

j U*(t,x,y) 

I 



= 1 



V*(t,x,y) = 



0 


for 


X > 0 


for T - X 2 < t ^ T - 


[''-“''xl 
aB J ’ 


1 - ^ 


for 


X = 0 







ay 
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and 



U*(t,x,y) = 1 
V*(t,x,y) = 1 



for T - 



A-av 



aB 



< t < T. 



(12): av^, < A and bv^, > B. 

A 1 

(1) If I S i{/v| + f(A-av^) = v^} - Tj. 

for y > 0 



U*(t,x,y) = I 
V*(t,x,y) = 1 



1 - r— for y = 0 
bx 



for 0 < t < T. 



( 2 ) 




for 0 < t < T - 



ay 



where 






(a) for y(t=T) > 0, 



= ■'I + ^{/b +2bAB{ 



A-av 



X 



aB 



- tY} - b} for y(t=T) = 0, 



Y Y 

where x is largest value of x such that y(t=T-x ) 

^ A e 

Y A 

and 0 < X < x’ = — , 

el aB 



= 0 






Y . 'x., 



(c) x^ = x| for y(t=T) = 0 when ^ ^1 ’ 



A final determination of the optimality of the extremal strategies still 
remains to be done in this case. Furthermore, the extremal strategy for 
Y is not unique. For convenience only one of two possibilities is given 
here . 
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and 



I U*(t,x,y) =1 ^ 

I V* 



V*(t,x,y) = 1 



for y > 0 
for y = 0 



for T - < t T 



Case (III): av A and < B. 

A ' 1 



( 1 ) 



” T il/v|+i(B-bv,) - \\ - •'i. 



U*(t,x,y) = 1 



I 

' V*(t,x,y) = ^ 



for X > 0 



1 - — for X = 0 
ay 



( 2 ) 



for 0 t T. 



If T > /v| + g(B-bv^) - Vjjl = T^, 



where 



U*(t,x,y) = 



V*(t,x,y) = 



r 



for y > 0 



1 - — for y = 0 
bx 



{“ 

^ 1 - — 
ay 



for X > 0 



for X = 0 



for 0 t T - T^, 






(a) for x(t=T) > 0, 



(b) + ^{/A2+2aAB{ 



B-bv, 



bA 



- T 



- a} for x(t=T) = 0, 



X X 

where t is largest value of x such that x(t=T=x ) = 0 
^ 1 G 

X 'V. 

and 0 < T < t’ = — r— - , 
el bA 



(c) T-, = t’ for x(t=T) = 0 when x^ > x’ , 

1 1 el 



•k 

A final determination of the optimality of the extremal strategies still 
remains to be done in this case. See also other comments in footnote for 
Case (12). 
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and 



U*(t,x,y) = I 





( ° 




for 


V-'-(t,x,y) = j 




r 

ay 


for 


: av,, A and 


bv 


_b. 


A 




1 


for 


U*(t,x,y) = < 




s 

bx 


for 




.0 




for 


V*(t,x,y) = . 




r 

ay 


for 



z = 0 



y - 0 



X = 0 



for T < < t < T. 



for 0 < t < T. 



6 . S ome Results for Case of Linearly-Decreasing-with-Time Returns frora 
G round Support . 

One criticism of the constant coefficient version (50) of the general- 
ization (1) of the tactical air-war game is that the criterion functional 
does not adequately reflect timeliness of ground support. Intuitively, we 
would expect that support of the ground forces is *'worth more” in the early 
stages of the ground war than later on. Thus, we are led to consider time- 
dependent ^returns” from ground support that decrease over time. Two cases 
of time-dependent returns from ground support that are of particular 
interest are 

(1) linearly-decreasing-wi th-time returns from ground support, 

(2) exponentiaily-decreasing-with-time returns from ground support. 



Interest in these cases was confirmed in informal discussions with Maj Ron 
Kronz, USAF. Such facets have been apparently explored in numerous runs 
of TAG CONTENDER. 
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Let us consider the case of (1) in whicli the Lanchester at I r i t ion-rn t(' 
coefficients for counter-*air operations are constants and the returns from 
ground support are time dependent (decreasing over time). Thus, we wii] 
consider the case in which the following hold: 



a(t) = a = constant, 
b(t) = b = constant, 

A(t) is given (decreasing) function of time, 
B(t)' is given (decreasing) function of time. 



For notational convenience we will again denote a as a and b as b. 
It is then convenient to re-state the problem as follows: 



maximize minimize { v x ( t^)-v y ( t -) + 
U V X f Y f 



[B(t)ux-A(t)vy ]dt} , 



with stopping rule: t^ - T = 0, 



subject to: 
(air-battle dynamics) 



dx 
d t 



r 



(l-v)ay, 




(l-u)bx. 



(144) 



with initial conditions x(t-O) = x^ and y(t=0) = y^ , 
and 

x,y r- 0 (State Variable Inequality Constraints), 

0 ^ u,v ^ 1 (Strategic Variable Inequality Constraints), 

where a and b are given constants and A(t) and B(t) are given 
f unc tions . 

In this section we wiJl consider the case in which both A(t) and 
B(t) are linearly decreasing functions of time for 0 < t < T. The situa- 
tion for A(t) is shown in Figure 6. We have then that 
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and 



A(t) = Aq - t 



cA„-A„> 
0 T 



B(t) = Bq - t 






T , 



wliere A^ > A^ and B^ > B^. 



(145) 



(146) 




Figure 6. Linearly-Decreasing-wi th-Time Returns from Ground Support A(t) . 
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We will now review the necessary conditions of optimality that we 
have developed for (144) in Section 3 before partially synthesizing the 
extremal strategies in one of four cases. We hope to give a more thorough 
study of this important problem in the future. Let us recall the swi tc iiing 
f unc t ions in backwards time x = T - t 

S^(t) = b(-q(T)) - B(t), (147) 



S^(t) = ap(i) - A(t) , 



( 148 ) 



where 



and 



A(t) = A(T-t) = A^ + t 



B(t) = B(T-t) = + t 



^0-^T 



( 149 ) 



0 T 



(15) 



Again, extremal strategies are given by (53) and (54) for x,y > 0. The 
differential equations for the switching functions are given by 



dS 

dr 






and 



dS 
\ 

dx 



b{A(x) + (10v*)S (x)} “ ~ for y > 0, 
V dx 



for y = 0 (for a finite interval of time). 



'' dA 

a{B(x) + (]-u*)S^(x)} - for x > 0, 



for X - 0 (for a finite interval of time), 



( 151 ) 



(152) 



with initial conditions 

S^^(x=0) = (bv^-B^) - bv^ and S^(x=0) == - av^, (153) 

where and must satisfy (60) and (61), respectively. It is also 

convenient to write (151) and (152) as 
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.1 

clT I 

^ 0 



b{A^+ T 



+ (1-v*)S^(t) - 



"O-'^T 



for y > 0, 



( 154 ) 



for y = 0 (for a finite interval of time), 



and 



dS j V " 



— Y“ + (I-u^)S^(t)} - 



A.-A^ 
0 T 



for X > 0 , 



dx 



lo 



(155) 



for X = 0 (for a finite interval of time) . 

For example. 



It is readily shown that singular solutions are impossible 
when x,y > 0 and 0 < v* <1, we have 

d^S 






= b 



Aq 



> 0 , 



and when v* is constant (i,e. the other cases in which v* = 0 or 1), 
we have 



d^S 



dx 



= b{v*^ 



A -A 

— 7 ^ — I + (l-v*)a[B^ + X 



+ (1-u*)S^(t)]}. 
d^S 



Observing that (l-u*)S (x) > 0, it is clear that — ■ -y • > 0 for all x 

u dx 

in the latter case, since it is a convex combination of positive quantities. 

X X 

On a constrained subarc on which x(t) = 0 for t < t < t we 

e 

have shown the following to be necessary conditions for optimality: 



v*(t) =1 TTY 

ay(t) 


for 


t^ < 
e 




(156) 


A 

rr 


for 


t^ . 
e 




(157) 




for 


t^ . 
e 




(158) 



and 



B(t)u* + (l-u*)b (-q(t) ) > — 

a 



A--A„ 
0 T 



( 159 ) 



Other convenient equivalent forms of (159) are 
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or 



a{B(t) + (l-u*)S^(t)l 



Aq 



maximum{ B(t) , 



b(pq(t))} 






(360) 



061) 



It is to be noted that (159) is no longer automatically satisfied on con- 
strained subarcs. Similar conditions hold on a constrained subarc on which 
y(t) = 0 for a finite interval of time. 

Concerning extremal end states, we have developed the following. 



PROPOSITION 5: If an optimal strategy for Y is to result 

in x*(T) = 0, then it is necessary that 

V — . 

X a 



PROPOSITION 6: 






If V < 

X a 



then an optimal strategy 
for Y leads to x*(T) > 0. 



PROPOSITION 7: If an optimal strategy for X is to result 

in y’^'(T) = 0, then it is necessary that 

B„ 



PROPOSITION 8: If v„ < 

1 b 



then an optimal strategy 
for X leads to y*(T) > 0. 



In synthesizing an extremal strategic-variable pair there are four 



cases to be considered: 



(I) 




( 1 ) 




(II) 




(2) Vy > -g- 



(2) V, 
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^ " ) 

^ 0 for 



A -A ^ 

+ (l-v*)S^(x) - 



^0-V 



for y > 0, 



(15A) 



y = 0 (for a finite interval of time). 



and 



dS_. I ”'“T 



dx 



a{B^+ T 

( 0 for 



«0-^T 



+ (l-u*)S (t)} - 
u 



A^-A^ 
0 T 



for X > 0, 



(155) 



X = 0 (for a finite interval of time). 

For example. 



It is readily shown that singular solutions are impossible 
when x,y > 0 and 0 < v* <1, we have 

d^S 



dx 



^ - K 

7- - h 



cAq 



> 0, 



and when v* is constant (i.e. the other cases in which v* = 0 or 1), 
we have 



d^S 



dx 



= b{v* 



(^] 



+ (l-v*)a[B^ + T 






; 



+ (l-u*)S^(x)]}. 
d^S 



Observing that (l-u*)S^(x) > 0, it is clear that ^ ® 

in the latter case, since it is a convex combination of positive quantities. 

X X 

On a constrained subarc on which x(t) = 0 for t < t < t we 

e 

have shown the following to be necessary conditions for optimality: 



v*(t) = 1 - 



ay (t) 
r < ay(t) 



P(t) = 



_ A(t) 



for 


t^ < 
e 


t 




(156) 


for 


t^<_ 


t 




(157) 




e 




i 


for 


t^ . 
e 


t 




(158) 



and 



B(t)u* + (l-u*)b (-q (t) ) > ^ 

a 



A-A„ 
0 T 



( 159 ) 



Other convenient equivalent forms of (159) are 
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or 



a{B(t) + (l-u*)S^(t)} ^ 




-A 



T 



maximum{ B(t) , 



b(pq(t))} 







( 160 ) 



(161) 



It is to be noted that (159) is no longer automatically satisfied on con- 
strained subarcs. Similar conditions hold on a constrained subarc on which 
y(t) = 0 for a finite interval of time. 

Concerning extremal end states, we have developed the following. 



PROPOSITION 5: If an optimal strategy for Y is to result 

in x*(T) = 0, then it is necessary that 

- T • 



PROPOSITION 6: If v < — , then an optimal strategy 

X a 



for Y leads to x*(T) > 0. 



PROPOSITION 7: If an optimal strategy for X is to result 



in y"^(T) = 0, then it is necessary that 
B^ 



v„ ^ 



PROPOSITION 8: If y 



then an optimal strategy 
for X leads to y*(T) > 0. 



In synthesizing an extremal strategic-variable pair there are four 



cases to be considered: 



(I) 



X 



< 




a 



( 1 ) 

( 2 ) 




(II) 




B-57 



We will give some initial results in the first of the above cases. 

In Case (11) : v < and v. , < — we have by Propositions 6 

X a 1 D 

and 8 that x*(T) > 0 and y*(T) >0 so that by (]53) we have 



Su(t=0) = bVy - < 0 and S^(x=0) = av^, - < 0 . (162) 



By the usual arguments. It is readily seen that 

u*(t) = 1 

(t) 



U*(T) = 1 
^ V* (t) = 1 



for T c [0,Tj^], 



(163) 



and x(x),y(T) > 0 for x e[0,x^). Using (162) and (163), the differentia] 
equations for the switching functions may be integrated to yield for 
0 < X < X, 



and 



S^(x) =- 



= t 






+ 



{bA^ 



(Bq-Bt) 



•^0-^T 



x^ + (aB„ - 



T j 



0 T 



}x + bv,^ - B^, 



}x + av^ - A^. 



(164) 



(165) 



Let T denote the (only) positive root of S (t=t ) = 0 and similarly 
u u u 

for T . Then we have 

V 



T = 
u 



= - {t 



Ao-At, 






Aq-At 



} + /{T 



Aq a^ 



"o-"t 



Aq A^ 



2T 



Bx-bv,^ 



Aq-At ^ 



(166) 



and 



T = 
V 



f B \ /'A -A >1 B~\ 

It — - — — — I 4- At — — 






J ta 



Ax-avx 






(167) 



The backwards switching time is then given by 

T, = minimum (t ,t ) < T. 

1 u V 



( 168 ) 
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It should be noted that in order for the switch to actually occur we require 
that ^ T. Omitting the rather tedious details, it is a straightforward 

matter to prove the following theorem: 



THEOREM 3: In order that e [0,T) for > bv^. 



(1) it is necessary that 



^o-"t 



< bA^, 



(2) it is necessary and sufficient that 



(a) 



^0-®T 



< bA^. 



and 



B -bv 

<») 

''o t' 



A similar theorem is readily proven concerning Thus, we see that in 

order for a switch in extremal strategies from (163) to occur additional 
conditions are required. This should be contrasted with the constant coeffi- 
cient case considered in Section 5 in which for the analogous situation (i.e. 

Case (II)). A switch in extremal strategies from (68) was guaranteed if 

A-aVx 

the duration of the battle was long enough (i.e. T > ). As an immediate 

a D 

corollary to Theorem 3, we have 



COROLLARY 3.1: If 



"'o'*! 



> aB^ and 






> bAp , then 



the solution to (144) with A(t) and B(t) 
given by (145) and (146) is given by 



u*(t) = 1 
v*(t) = 1 



for 0 < t < T. 
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It should be noted that condition (2b) of Theorem 3 may be written as 

B(t=0) - bVy 

< 



(169) 



bA 



where 






A(t)dt. 



Shortness of time precludes further results for this problem. The author 
hopes to give further details in the future. 



7 . S ome Results for Case of Exponentially-Decreasing-with-Time Returns from 
Ground Support . 

In this section we wiJl briefly consider the case in which both A(t) 
and B(t) are exponentially decreasing functions of time for 0 < t < T. 

We have then that 

A(t) = A^ e■“^ (170) 

and 

B(t) = Bq (171) 

where a, 3 > 0 . 

We will now review the review the necessary conditions of optimality 
that we have developed for (144) in Section (3) before partially synthesizing 
the extremal strategies in one of four cases. Again, we hope to give a more 
thorough study of this important problem in the future. As before, the 
extremal strategies are given by (53) and (54) for x,y > 0. The differen- 
tial equations for the switching functions are given by 
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jg / b(l-v*)S^(T) + (bA^e - (PB^e for y > 0, 

— = < 
dT ) 



for y = 0 (for a finite interval of time). 



and 



0 72) 



dS ( a(l-u*)S^(T) + (aB^e - (“A^e for x > 0, 

— ■ = I 

dr 1 

0 for X = 0 (for a finite Interval of time), 

with initial conditions 



073) 



-RT -fiT 

Su(t= 0) = (bv^-B^e ) - bv^ and S^(t= 0) = (av^-A^e ) - av^, (174) 

where and must satisfy (60) and (61), respectively. 

X X 

On a constrained subarc on which x(t) = 0 for t < t ^ t , we 

e 

have shown that (156) through (158) are necessary conditions for optimality 
and also 

A 

B(t)u* + (l-u*)b (-q (t) ) > e (175) 

a 

Other convenient equivalent forms of (175) are 



a(B(t) + (l-u*)S^(t)} > “A^e'”'^, 



or 



aA 



maximum (b ( t) ,b (-q (t) ) } > e 

a 



0 -at 



(176) 



(177) 



As in the previous case, it is to be noted that (175) is no longer auto- 
matically satisfied on constrained subarcs. Similar conditions hold on a 
constrained subarc on which y(t) = 0 for a finite interval of time. 

Concerning extremal end states, we have developed the following. 



PROPOSITION 9: If an optimal strategy for Y is to result in 

x*(T) = 0, then it is necessary that 



> V 

V > — 

X a 



-a-p 
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A e 

PROPOSITION IQ: If v < — 

X a 



-aT 



, then an optimal strategy for 



strategy for Y leads to x*(T) > 0. 



PROPOSITION 11: If an optimal strategy for X is to result in 

y*(T) “ 0, then it is necessary that 
B e-ST 

' b 



^ -3t 

PROPOSITION 12: If < , then an optimal strategy 

for X leads to y*(T) > 0. 



In synthesizing an extremal strategic-variable pair there are four 
cases to be considered: 



(I) V^< 



V 



-6T 



(II) 



-aT 



-3T 



^0^ 

(1) v^<-^ 

p -BT 

( 2 ) 



^0^ 

(1) v^<-g 



-3T 



( 2 , 



-BT 



We will now give some initial results in the first of the above cases. 



In C ase (II): v^ < 



^0^ 



-aT 



^0^ 



-3T 



and < r~ , we have by Proposi- 

— — Y b — 



tions 10 and 12 that x*(T) > 0 and y*(T) > 0 so that by (174) we have 

S^(t= 0) = bv^ - Ege”^^ < 0 and S^(t= 0) = av^ - A^e < 0 . (178) 

By the usual arguments, it is readily seen that 



u*(t) = 1 



/ for T E (0, T, ] , 

^ V* ( T ) = 1 



( 179 ) 
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and x(T),y(x) > 0 for x c [0,Xj^). Using (178) and (179), the differential 



equations for the switching functions may be integrated to yix'ld for 
0 < X ^ X, 



S^(x) = 



bA^e 



-aT 



0 



and 



S^(t) . 



“V 



a j 
6T 



e - (B^e 



A„e 



“3T, Bt ^ , f “0 
- )e + b v„ - 



-aT 



I Y 



(180) 



6x .. -aT. ax ^ 
e - (AqC )e + a 



R 

BQe 

'"x 6 



(181) 



Let T denote the smallest positive root of S (t=t ) = 0 and similarly 
u u u 



for T^. Then the backwards switching time is given by 



X. = maximum (x ,x ) ^ T. 
1 u V 



(182) 



We will now give some results for the following two cases: 

(1) a = 3, 

(2) 6 > a. 



In Cas e (1) : a = 3 > in order for x^ to exist such that ^ 



wc must have 



“o " ®''o’ 



(183) 



and then 



1 , b(A(,-Be%)' 

^u a ^ ^ bA„ - RR J 



‘0 



( 184 ) 



Omitting the details for now, the following theorem may be proven: 
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THEOREM 4: In order that r e [0,T) for a = R and 

u 




(1) it is necessary that — (t=0) < bA(t=0) (where 
A(t) and B(t) are given by (170) and (171)) 



(2) it is necessary and sufficient that 



(a) -jf(t=0) < bA(t=0), 



and 



(b) { } < T, 



B(t=0) - bv 



bA 



where 




0 



A similar theorem concerning may be proven. The reader should note 

that Tlieorems 3 cmd 4 are basically identical. Thus, there is great simi- 
larity (at least for the results so far obtained) between the cases of 
J inearly-decreasing-wi th- 1 line returns from ground support and exponentially- 
decreasing ones. As above, we see that in order for a switch in extremal 
strategies from (179) to occur additional conditions in addition to the war 
being of sufficient duration are required. This should be contrasted with 
the constant coefficient case (i.e. Case (II)) considered in Section 5. 

As an immediate corollary to Theorem 4, we have 



COROLLARY 4.1: For the case in which a = B, if 



~“(t=0) > aB(t=0) and 'Tr(t=0) ^ bA(t=0) 
d t d t 



then the solution to (144) with A(t) and 
B(t) given by (170) and (171) is given by 




u*(t) = 1 



for 0 < t < T. 
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Case (2) : 3 > o t > the following theorem may be shown to hold. 



THEOREM 5: In order that e [0,T) for 3 > cx and 

B(t=T) > bv^, it is sufficient that 

(a) -jf(t=0) < bA(t=0), 

and 

B(t=0) - bv 

(b) { } < T. 

bA 



Shortness of time precludes further resu3 ts for this problem. The author 
hopes to give further details in the future. 

^ • Discussion . 

In this appendix we have applied the theory of differential games 
(including the theory of state variable inequality constraints developed 
in Appendix A) to the study of optimal air-war strategies. We have con- 
sidered a generalization of the tactical air-war game studied by Isaacs 
[7] and others [1], [4], [6], [16], [24]. We saw that when one considers 
salvage values for aircraft, time-dependent returns from ground support, 
and time-dependent Lanchester attrition-rate coefficients, previous solution 
methodology for the tactical air-war game can result in non-optimal strate- 
gies. For example, during the early phases of the air war, it may not be 
optimal to annihilate enemy aircraft whenever possible. We a].so saw that 
when there is a residual value for aircraft at the "end" of the war, optimal 
strategies may be appreciably different than those for the special case 
previously considered. 

We developed a (fairly) complete solution to the generalization of 
the tactical air-war game for the case of constant coefficients. Some work 
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still remains to be done in one special case. We propose this to ONR as a 

future research task. We also partially developed extremal strategies in 

two cases of time-varying (decreasing) returns from ground support (both 

linearly-decreasing returns and also exponentially-decreasing ones). In 

these cases we have shown that optimal strategies for both sides may be to 

always concentrate on supporting the ground forces regardless of how long 

the conflict lasts . For the case in which av < A(t=T) and bv < B(t=T), 

X 1 

we showed that an additional condition had to be satisfied (namely, 
dB 

— (t=0) < bA(t=0)) in order for it to be possible for a switch in X*s 
optimal strategy to occur. (It should be noted that this latter condition 
is always satisfied for the case of constant returns from ground support.) 

The results of this appendix are not only of intrinsic interest but 
are also useful for checking the computational adequacy of Lagrange dynamic 
programming, which has been proposed as a compational method for time-sequential 
combat games [13], [14]. It should be pointed out that there may be severe 
computational difficulties in using Lagrange dynamic programming when the 
solution to a time-sequential combat game has multiple extremals (as is 
frequently the case (see [19])). 

We think that an understanding of the parameters which affect optimal 
air-war strategies is essential. In particular, the effects on optimal 
strategies of considering salvage values for aircraft, time-dependent returns 
for ground support, and time-dependent Lanchester attrition-rate coefficients 
are not entirely understood at this time. We feel that an analytic approach 
like that employed here is very useful for discovering cause-effect relation- 
ships between optimal strategies and modelling assumptions. We have laid 
the foundations for future research in this area and recommend such future 
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study to ONR. Of particular importance, is to determine whether or not 
the basic model is adequate. A basic shortcoming of the model (1) is that 
it does not evaluate air-war tactics within the context of ground-war 
objectives, llie model suboptimizes. In the next appendix we briefly 
C(')nsider a model that does evaluate air-war tactics within the context 
of ground-war objectives. 
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APPENDIX E: Some Problems for Future Study. 



1 . Introductio n. 

In this appendix we will briefly describe several important problems 
for possible future research. These problems are all extensions of work 
reported both here in this report and elsewhere [9], [10]. 

As we have pointed out recently [11], when we began our research, the 
state-of-the-art of deterministic control theory (at least that for applica- 
tions to operations research (see [12])) was not sufficient to allow routine 
solution of even the simplest Lanches ter-type optimal control problem. Our 
research approach has been to consider a sequence of idealized models of 
increasing mathematical complexity and also increasing operational, realism. 

At this point in time, most of the simplest Lanches ter- type tactical 
allocation structures have been fairly thoroughly studied. Some theoretical 
results in optimal control theory have resulted from this examination [12], 
[13]. To solve the problems at hand that we propose, we would build upon 
the knowledge gained by solving numerous particular examples. 

The problems that we will elaborate upon below may be referred to as 

(1) fire support for several ground units, 

(2) incorporation of logistics considerations, 

(3) effect of the form of the criterion functional in combat 
optimization problems. 

All the above problems are quite important. The first is important because 
it is a closer approximation to the type of decision that a commander must 
make in allocating supporting fires to aid various friendly units than we 
have previously considered. The second is important because it incorporates 
logistics considerations into Lanches ter- type equations of warfare. As far 
as Navy missions are concerned, this is very important, since sea-based 
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supply and re-supp]y is a principle Navy combat support mission. The third 
is important because it provides more insight into the relationship between 
the structure of optimal strategies and the nature of (system) objectives 
(here nonlinear) . 

As we first pointed out in [11], for the purposes of military analysis, 
it is convenient to consider that there are three essential parts of any 
dynamic combat optimization problem: 

(a) the decision criteria (for both combatants), 

(b) the model of conflict termination conditions (and/or unit 
breakpoints) , 

(c) the model of combat dynamics. 

Our proposed problems (1) and (2) above fall into the category of research 
on the effects on the structure of optimal strategies of aspect (c), whereas 
(3) falls into the category of research on (a). It is this central analysis 
aspect of the combatants' decision criteria that the author believes is the 
weakest link in the chain of assumptions in current analyses. Pugh and 
Mayberry [4] have recently proposed methodology for this important topic 
of measures of effectiveness in dynamic combat optimization problems. 

However, they do not explore the cons equences of various functional forms 
of the criterion functional. This is the objective of studying problem (3). 

Finally, it should be pointed out that all the problems considered 
below are stated in their simplest form. This is because of our research 
objective of studying solution properties, an objective whose attainment is 
facilitated by the availability of complete, explicit analytic solutions. 

More realistic versions of these problems are readily formulated. However, 
mathematical tractability is sacrificed when this is done. Moreover, our 
results for such idealized problems provides guidance for developing numeri- 



cal solution algorithms for more realistic campaign models. 
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2 . Fire Support of Several Ground Units . 

Consider combat between heterogeneous X and Y ground forces 
(infantry). Contact exists between these forces along the l\3rward Edge 
of the Battle Area (FEBA) . The X ground forces are composed of two units, 
denoted as X^^ and X^, and similarly for the Y ground forces. It is 
assumed that the X^ forces are only in contact (i.e. combat) with the 
Y^ forces and similarly for X^. In each of the two "one-on-one” battles 
both combatants undergo a "square-law" attrition process (see [6]) from 
enemy (small arms) fire. 

The problem facing the X commander is to determine the "best" dis- 
tribution of the fire of his artillery, denoted as U, over the enemy units, 
i.e. Y^ and Y^. The artillery U fires at a constant rate into the 
(constant) area of an enemy unit without feedback as to the destructiveness 
of this fire. Tliis situation is shown in Figure 1. X*s decision criterion 
is the net worth of survivors (with a linear utility for survivors ) . For 
the purposes of tliis report, discussion of the stopping rule (i.e. battle 
termination model) is not essential and is omitted. 



Mathematically, the problem may be stated as 

T T 

maximize {v x(T) - w y(T)}, 



% % 
dXj 

subject to: , 

dx^ 

dT = -V2’ 



( 1 ) 






'Pugh and Mayberry [4] suggest using the ratio of aggregated force value. 
For our purposes here the exact form of the criterion functional is not 
essential . 

See [2] for methodology for the development of these linear utilities. 



Infantry 




Figure 1. Diagram for Fire Support of Several Ground Units. 
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(l-4>)c y 



2 ^ 2 ’ 



with initial conditions 



x^(t=0) = and y^(t=0) = for i = 1,2 



and 



^ (State Variable Inequality Constraints), 



x^.yi >y 



0 < (() < 1 



(Control Variable Inequality Constraints) 



where 

x^(t) [for i = 1,2] is the number of infantry at time t, 

similarly for y^(t), 

a. [for i = 1,2] is the constant (Lanchester) attrition-rate 

coefficient (the rate at which one unit can destroy X^), 

similarly for b^, 

c. [for i = 1,2] is the constant (Lanchester) attrition-rate 

coefficient for the effectiveness (area fire) of U artillery 
against Y^, 

I = (v^.v^). 

is the value of one surviving X^ unit, 
similarly for w. 

To be precise, in the above formulation we have 



i 

dt 



dx 



-a.y. for 
1 1 



x^ > 0 , 



0 



for X. = 0 
1 



and 
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dt 



J 



-b^Xi 



<l>ciy] 



for 



y-^ > 0 , 



for = 0. 



Moreover, we would begin by considering the case in which x^, 
and y° are such that (T) .x^d) ,y^ (T) .y^CT) > 0. 

It should be clear that the above problem (1) is important because 
it considers a basic aJ location problem for artillery fire. With minor 
modifications the model (1) may be extended to other cases of interest: 

(a) let b^ ^2 ~ ^ ^ prescribed duration battle attack scenario 

(see Appendix D), (b ) for an amphibious assault one adds replacements for 
the and forces. 

Instead of being called "artillery," the fire support units U may 
be called "Naval guns." Of particular relevance to Navy problems in the 
determination of the optimal allocation of Naval gunfire in amphibious 
assaults . 

Finally, it seems appropriate to discuss a technical difficulty of 
solving the above problem (1). As are all Lanches ter-Lype fire distribution 
problems, the above problem (1) is a singluar problem of optimal control , 
since - 0 (see [7]), where H denotes the Hamiltonian. The problem 

d(|) 

becomes particularly difficult when a singular solution exists in x - ^ 
space (see [1]). A preliminary investigation has shown that this is the 
case for the problem at hand- It should be pointed out that we have encoun- 
tered such difficulty previously in our research reported here (see Appendix 
D) . 
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3 . A Lanchcs ter-Type Optimal Control Problem with Logistics Considerations , 

In this section we will describe a simple model which, nevertheless, 
can generate some insights into tactics used by General George S. Patton, 

Jr., in World War II. Consider combat between X and Y forces. Part of 
the X forces (denoted as X^) can be kept in reserve and consequently do 
not consume supplies as rapidly as the combatant forces (denoted as X^^) do. 
It is assumed that the effectiveness of X^ against Y depends upon the 
amount of supplies (denoted as S) that they have. For the problem under 
consideration, this effectiveness will be represented by a Lanchester attri- 
tion-rate coefficient b^ = b^^(S). It seems reasonable to hypothesize that 
b^ (S) is a concave function of S with certain other appropriate properties. 
Further research should be done on this important topic. For convenience in 
our initial investigation, we would assume that b^ is constant but that 
S > 0. We further assume that due to ’’pipeline” capacity there is an upper 
limit to the rate at which supplies can be replenished. 

The decision variable under the control of the X commander is the 
rate of reinforcing (or withdrawing for u(t) negative) the X^^ forces. 

This situation is shown in Figure 4. Tlie objective of the X commander 
is to maximize the net worth of survivors (considering linear utilities). 

In mathematical terms, the problem may be stated as follows. 



maximize {px^^(T) + qx 2 (T) 



u(t) 



dt 

dx. 



ry(T)} with 
-ay + u, 



dt 



T unspecified. 



^ = 
dt 



-bx^. 



dS 

dt 



-cx^ + P, 



subject to: 



( 2 ) 



E-8 




Figure 2. Diagram of Supply-Constrained Combat Problem. 



with initial conditions 

x^(t=0) = x" for i = L,2, y(t=0) = y^, S(t=0) = S^, 

and 

^ (State Variable Inequality Constraints), 

-W < u < R (Control Variable Inequality Constraints) , 

and with the stopping rule 

(a) x^(T) = 0, 

(b) y(T) = 0, 

or (c) S < 0, 



where 



c denotes the rate of consumption of supplies S by one 
comb a tan t, 



R,W > 0, 
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P denotes the ’’pipeline” capacity, 

S denotes the supply level of the forces, 

u(t) denotes the rate of reinforcement of X forces by the 
X^ forces (negative quantity denotes withdrawal), 

and other notation is similar to that used in the statement of problem (1). 

A preliminary examination of problem (2) has yielded an Interesting 
result: if supplies are to become a constraining factor for the X forces 

later in combat, then the optimal tactic is to ’’overcommit” forces early in 
the campaign (i.e. forces are being withdrawn at the moment that the supply 
constraint becomes active). Of particular mathematical difficulty is the 
presence of a second order SVIC (i.e. S > 0) (see [8]) in this problem. 

The only previous work considering logistics aspects apparently is 
by Moglewer and C. Payne [3]. Our above problem (2) may be extended to 
yield insights for policy planning regarding ’’pipeline capacity” (logistics 
capacity), pipeline protection, pipeline interdiction, etc. 

4 . Form of the Criterion Functional in Time-Sequential Combat Optimization 
Problems . 

In all our past research, we have with one exception (see Appendix D 

* 

of this report) always considered a linear utility for survivors in the 
criterion functional. It is of interest to examine how the valuation of 
survivors affects the structure of optimal strategies. It seems appropriate 
to begin such an investigation by considering the simplest problem possible. 
Thus, we will consider a version of the ’’Tactical Air-War Campaign” (see 
[10] and Appendix B of this report). 

See [2] for methodology for the , development of such a linear utility. 
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Mathematically, the problem may be stated as follows. 



maximize {Q(x(t ,.) ,y (t ,) ) + 
<^(t) 



[x(l-((i)-y]dt}, 



0 



with stopping rule: t^ - T = 0, 

dx 



subject to: 
(air-battle dynamics) 



dt 



= r, 



^ = s - (j)bx, 



with initial conditions 

x(t=0) = Xq and y(t=0) = y^^, 

and 

x,y > 0 (State Variable Inequality Constraints), 

0 < <{) < 1 (Control Variable Inequality Constraints) , 

where 

x(t) and y(t) denote the numbers of X and Y aircraft, 
respectively, at time t, 

b denotes a Lanchester attrition-rate coefficient (the rate 
at which one X aircraft shoots down Y aircraft), 

r and s denote replacement rates, 

4>(t) denotes the fraction (at time t) of total X aircraft 
which fly counter-air missions and hence shoot down Y 
aircraft , 

and Q(x(t^) ,y (t^) ) denotes the (salvage) value of X and Y 

aircraft at the end of the planning horizon. 



It is of interest to examine the dependence of the structure of the 
optimal allocation policy for X aircraft upon the nature of G(x^,y^), 
where x^ denotes x(t=t^), etc. It seems reasonable on military grounds 



to require that 
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iQ 



9x 



f 



> 0 



and 






< 0 



for all - f*- 



Furthermore, one might postulate either of the following functional forms 
for Q(x^,y^):’' 



or 



(a) Q(x^,y^) = F(x^) - G(y^), 



F(x ) 

(b) Q(Kf,yp . . 



It Is of interest to study cases in which F (and/or G) is a 



or 

or 

or 



(A) concave function, 

(B) convex function, 

(C) quasi-concave function, 

(D) quasi-convex function. 



To consider a concrete example, if G(y^) were a concave function, it might 
look like that shown in Figure 3. One possibility for an analytic representa- 
tion of the function shown in Figure 3 is 

3q -“Yf 

G(yf) = (1-e h. (4) 

After the dependence of the structure of the optimal allocation 
policy upon the nature of the terminal return G(x^,y^) has been studied 
for the above problem (3) (preliminary analysis indicates that this problem 
is analytically tractable), it would seem appropriate to consider a problem 
like the Isbell-Marlow fire distribution problem (see [5], [6], [8], or [10]), 
which is at the next level of analytical complexity. Such a research program 

Pugh and Mayberry [4] have suggested using the ratio of aggregated force 
values . 



E-12 




Figure 3. Decreasing Marginal Value for Survivors. 

would lead to a better understanding of the effects upon (optimal) military 
decision making of the valuation of (system) objectives. This in turn would 
result in a better understanding of the optimization of combat dynamics. 
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